Does the world need another graph database?

[Cayle]

So my flirtation with refactoring Angela to use Neo4j is at an end. Its not that graphs don’t look like my silver bullet for the kind of gameplay that I want in magicus occultus. It’s just that while 60% of Angela’s special features map directly to existing property graph databases, there is some stuff that Angie can do that I’d need to re-implement. Thinking about it more deeply, I realized that if I took out the asynchronous app engine stuff, Angela is – at its core – itself almost a property graph DB. Moreover, it has a few tricks up its sleeve that make it unique.

• It can be used schema-less, or a schema can be used for extra functionality. This schema is a two level model/metamodel (called meme and metameme) affair. So at runtime, an individual orc is an entity, but the template from which all new orcs are instantiated is a meme (model) and the high level definition of mobs is the metameme (meta-model).
• Angela can do wildcard traversals. I have X and suspect that there is a Y in the connect network; but don’t know the exact path. E.g. I have a character’s inventory and some inventory items are themselves containers. If I want to check for a quest item in character X’s backpack, I can do such a wildcard traversal to look for it.
• Angela can do graph traversals based on the metameme and meme of an entity.

I’m missing a few things:

• Relationships need to be first class citizens. Right now, entities are first class and relationships are just glue.
• Relationships need to be directional. Right now, they actually are; but when queried I actually conflate both directions into a binary “link”.
• Relationships need to gain properties.
• For really sophisticated traversal using these relationships during , it should be able to parse the Gremlin query language.
• I need some persistence mechanism; probably be doing lazy writes to a relational DB.

None of these things are – by themselves – earth shattering changes. I already roughly know what needs to be done to implement them all. Then it occurred to me. All of this database stuff takes place in two modules. These two modules are not solely dedicated to the DB; after all, there are some core app-engine bits mixed into both modules, but I could – in principle – do a bit of refactoring and separate the DB into a separate module.

The question is, does the world need an MIT licensed graph DB?

[Xaos]

On one hand, it seems wasteful to have core parts in DB modules, and could hamper the performance of the DBs, but if they are small it could def. be a good thing to tie the core parts into the DB modules. But if you get to the point where the core parts being in the DB module, you're probably willing to just refractor your engine.

[Cayle]

I did not set out to write a DB. I set out to write an application framework for running MMOs that used "highly modular and reusable content". I'd essentially arrived at building a graph database to solve this problem and only realized what I had - a graph database within my game engine - later on. About 70% o the graphDB code is in a plugin package (specifically the DNA.py module of the AngelaRML package) and about 30% resides in Angela's Engine.py module (Engine.py is essentially the kernel). The stuff in the plugin was there because I had been intending to migrate to a "real database" sometime and so wanted this to be removable. The bits in the Engine.py module were a concession to Angela's highly threaded nature. I needed certain bits (the actual repositories for entities and links) to be singletons. So moving everything to a graphDB module that is imported a a normal import by Engine.py is a

I may never finish Magicus Occultus. It is a fairly ambitious project with one developer. I don't expect anyone ever to use the engine that it is built on. Angela is a complex beast with many moving parts and even just getting it running is tricky. The graph DB... someone may actually be able to use that.

[Xaos]

I did not set out to write a DB. I set out to write an application framework for running MMOs that used "highly modular and reusable content". I'd essentially arrived at building a graph database to solve this problem and only realized what I had - a graph database within my game engine - later on. About 70% o the graphDB code is in a plugin package (specifically the DNA.py module of the AngelaRML package) and about 30% resides in Angela's Engine.py module (Engine.py is essentially the kernel). The stuff in the plugin was there because I had been intending to migrate to a "real database" sometime and so wanted this to be removable. The bits in the Engine.py module were a concession to Angela's highly threaded nature. I needed certain bits (the actual repositories for entities and links) to be singletons. So moving everything to a graphDB module that is imported a a normal import by Engine.py is a

I may never finish Magicus Occultus. It is a fairly ambitious project with one developer. I don't expect anyone ever to use the engine that it is built on. Angela is a complex beast with many moving parts and even just getting it running is tricky. The graph DB... someone may actually be able to use that.

Oh, I think I misunderstood your OP.

I think your graphDB could easily be used, especially if it can be easily tied to a complex engine.

[Jackolantern]

If it does some things radically different from currently available dbs, it may be worth it. If it is mostly like existing graph dbs with some additional features, I am not sure it is worth taking the huge detour on your project to get it in a publicly-usable form. I have not used a graph db, so I am not sure if it meets that requirement or not from the features you mentioned.

[Cayle]

That's the crux. I don't feel like just throwing it out there just because. On the other hand, I'm not super exited by anything out there. I'm going to refactor Angela in any case, to clean up its architecture and I can clean up the standard schemas in a bit by adding reference directions and properties. Whether I go to the last bit of effort and release the DB as its own thing or not is something I can decide later.

Edit: "to refactor Angela in any case" and "the last bit of effort ". The "f" key on my keyboard is a bit dodgy these days.

[Jackolantern]

Sounds like a plan