Neo4jClient 4.0

After what probably seems like ages to you (and indeed me) Neo4jClient 4.0 has finally left the pre-release stages and is now in a stable release.

Being a major version change, that means there are breaking changes, and you should be in the process of testing stuff before you just use it. Having said that, the changes which are there hopefully make sense, and will make it better in the long run.

It should be noted, that some of the things done are only applicable to Neo4j 4.x servers, and if you’re not using .NET Core, or not using Transactions – staying with the 3.2.x release of the client will be fine for now.

OK. Onto the changes, we have 4 broad categories, Breaking changes, General changes, Additions and Removals. I’ll try to demo as many of them as possible with code, but for the self-explanatory ones – well – I probably won’t 😮

Breaking

These are ones which will require you to do some code changes, how much will vary depending on your codebase.

Async Only

There are no ‘Sync’ methods anymore, decision wise – .NET code has been increasingly moving towards Async, and the client has always supported it. If there is enough clamour – I will look into re-adding the Sync wrappers, but at the moment, less-code = less to maintain.

No MSDTC (TransactionScope) Support

The 3.x versions of the client used TransactionScope to provide transaction support – which had a nice benefit of being able to support MSDTC. But, as I’m sure many know, it also prevented .NET Core support for transactions due to the availability of TransactionScope and the supporting classes in Core. With .NetStandard 2.0 TransactionScope was added, but the supporting classes not, and whilst now they are available in NetStandard 2.1 – I don’t want to push the minimum requirements for the Client to NetStandard 2.1. As a consequence, the decision has been made to remove support for TransactionScope.

Long term – the aim is to have an ITransactionManager that you can inject into the Client to allow you to provide a home rolled TransactionScope manager if you really want it. This doesn’t exist – and won’t for a while as because as far as I’m aware, there was only one company using it, and even they said they were moving away from it.

Neo4j Server 3.5+ only

Neo4j don’t support server versions lower than 3.5, and whilst the GraphClient should work with any of the 3.x servers (which support transactions), the BoltGraphClient will only work back to 3.5.

Personally – I’m not to worried about this, aside from Transactions all of the additions to the client wouldn’t work on older server versions anyhow, as they didn’t exist. Basically – if you use an older version of the server – use the older client!

Other

There are other changes that will come with the ‘Removals’ section below, but I thought I’d write about them there!

General Things

So some general changes here about the client, you might find it interesting, you might not :/

NetStandard 2.0

Thanks to a PR from tobymiller1 – (https://github.com/tobymiller1) the client is back to being just one project, and targets NetStandard 2.0

URI Schemes

The client now supports all the schemes Neo4j does, so neo4j, neo4j+s, neo4j+sc and the bolt equivalents.

Transactions

I wasn’t sure if this should be Addition or Removal or … so I’ve gone with ‘General’. Transactions needed to be changed, as I noted above, and as part of the ability to target Neo4j 4.x we needed to support Multi-tenancy – that’s multi-databases within one server.

Let’s have a look at some examples:

using(var tx = gc.BeginTransaction(TransactionScopeOption.Join, null, "neo4jclient"))
{
    await gc.Cypher.Create("(n:Node {Id:1})").ExecuteWithoutResultsAsync();
    var insideResults = await gc.Cypher.Match("(n:Node)").Return(n => n.As<Node>()).ResultsAsync;
    insideResults.Dump("In the Transcation");

    await tx.RollbackAsync();
}

var outsideResults = await gc.Cypher.Match("(n:Node)").Return(n => n.As<Node>()).ResultsAsync;
outsideResults.Dump("Out of the Transcation");

Which gives out:

Let’s take the code apart a bit.

using(var tx = gc.BeginTransaction(TransactionScopeOption.Join, null, "neo4jclient"))

Transactions are IDisposable – so you ideally should be using them with a using statement, but if not, remember to dispose()! In this version, you have to supply the TransactionScopeOption and a bookmark (the null) in this case, to be able to define the database ("neo4jclient"). That’ll probably be simplified later…

Next, we’re just executing Cypher, as we’ve done plenty of times before, and as I’m using LinqPad I can call .Dump() on any output to get the results (we saw in the picture). So I can access the stuff I’ve put in the database, within the transaction.

I then .RollbackAsync() the transaction, the effects of which can be seen by the second .Dump() I execute showing nothing in the database. Also note, with the second Results query – I have to provide a WithDatabase parameter, else I would be querying the default database.

You HAVE to .CommitAsync() a transaction for it to be committed, if I didn’t have the .RollbackAsync()) call, and just let the tx be Dispose()-ed by the using statements closing, it would automatically be rolled back. In the code above, the .RollbackAsync() call is redundant.

Write transactions are on one database only, an attempt to write to another database (using WithDatabase or Use) will result in a ClientException – nb. you can use Use to read from another database in a transaction.

Additions

New stuff for you to use! After each heading will be a list of the versions of Neo4j Server that the additions will work on.

DefaultDatabase [4.x]

Multi-tenancy brings a load of new things to the Neo4j world, but how can you use them from Neo4jClient?? First off let’s talk about DefaultDatabase. This is a property of the GraphClient/BoltGraphClient itself. By default it’s set to ‘neo4j’ (which is the default the server has), but you can set it to any other database, and every query will be against that one.

var gc = new GraphClient(...){DefaultDatabase = "neo4jclient"};

.WithDatabase() [4.x]

But what if you want to run just one of your queries (or more!) against another database? That’s where WithDatabase comes in. WithDatabase is a per query setting to choose the database a query will run on:

gc.Cypher.WithDatabase("neo4jclient").Match(...)

CreateDatabase (system) [4.x]

Want to create a database? Sure, you can do that! This needs to be executed against the system database, and you have to use the WithDatabase call do to that:

gc.Cypher.WithDatabase("system").CreateDatabase("neo4jclient", true).ExecuteWithoutResultsAsync();

Wait? What’s this true parameter? It’s the ifNotExists parameter which means that we will only create the database if it doesn’t exist.

StartDatabase (system) [4.x]

Well, now we’ve created our database, we need to start it:

gc.Cypher.WithDatabase("system").StartDatabase("neo4jclient").ExecuteWithoutResultsAsync();

If you run this on an already started database, it’ll do nothing!

StopDatabase (system) [4.x]

We’ve started it, now let’s stop it. No surprises here:

gc.Cypher.WithDatabase("system").StopDatabase("neo4jclient").ExecuteWithoutResultsAsync();

Again, as per the StartDatabase method, stopping a stopped database does nothing.

DropDatabase (system) [4.x]

Dropping a database is the quickest way to ‘truncate’ your entire database without resorting to stopping the server. As with CreateDatabase we have two parameters here, the first is the database name, the second is dumpData (default false). If dumpData is false then calling DropDatabase will delete all the data files, if it’s true the data will first be dumped to the location the server has specified (see the documenation for more information)

gc.Cypher.WithDatabase("system").DropDatabase("neo4jclient", true).ExecuteWithoutResultsAsync();

If you run this on a database that has already been dropped – you will get a FatalDiscoveryException thrown, if you want to avoid that, you need to use:

DropDatabaseIfExists (system) [4.x]

Pleasingly this is the same as DropDatabase just that you won’t get the exception if the database doesn’t exist. You have the same option to dumpData or not. It’s all up to you!

gc.Cypher.WithDatabase("system").DropDatabaseIfExists("neo4jclient", true).ExecuteWithoutResultsAsync();

Use [4.x]

Use() allows you to use a database within a query or query part. It means you can avoid having to use WithDatabase, or you can use it within a query that is using another database. Whaaaat?! Confusing?! Yes!

gc.Cypher.Use("neo4jclient").Match("(n)").Return(n => n.As<Node>())

Gives us:

USE neo4jclient
MATCH (n)
RETURN n

BUT you can’t do this for things like the system database calls above, you have to use the WithDatabase clause for that. Use is particularly Useful (ha!) for Fabric use cases.

WithQueryStats [3.5,4.x]

With 3.x when you executed a query, you couldn’t tell what that query had done, in fact, all you could know was that you executed a query (especially if it was a ‘non results’ one). Now you can use the .OperationCompleted event to get the stats of your query:

void OnGraphClientOnOperationCompleted(object o, OperationCompletedEventArgs e)
{        
    e.QueryStats.Dump();
}
gc.OperationCompleted += OnGraphClientOnOperationCompleted;
await gc.Cypher.WithQueryStats.Create("(n:Node {Id: 10, Db: 'neo4j'})").ExecuteWithoutResultsAsync();
gc.OperationCompleted -=OnGraphClientOnOperationCompleted;

Which will give you a QueryStats object that looks like:

Check that out! I added a new Label, 1 Node and 2 Properties!

This isn’t on by default, as it sends back more data over the wire, and if you don’t need it (which so far 100% of people haven’t) then it’s optional!

Neo4jIgnoreAttribute [3.5,4.x]

From a PR by @Clooney24) This will provide the ability to ignore properties for the BoltGraphClient as well as the GraphClient.

So, let’s have our class:

public class Node 
{
    public string Db {get;set;}
    public int Id {get;set;}

    [Neo4jIgnore]
    public string Ignored {get;set;}
}

We’ve defined the Ignored property with the [Neo4jIgnore] attribute, so now when we insert the data:

var node = new Node { 
        Id = 11, 
        Db = gc.DefaultDatabase, 
        Ignored = "You won't see this!" 
    };
await gc.Cypher.WithQueryStats.Create("(n:Node $newNode)").WithParam("newNode", node).ExecuteWithoutResultsAsync();

and then pull it back:

(await gc.Cypher.Use("neo4j").Match("(n)").Return(n => n.As<Node>()).ResultsAsync).Dump("Ignored");

We can see that Ignored is null:

And this isn’t just because we’re ignoring bringing it back, but it’s not in the database either:

Removals

These are all things that have been removed, largely, they were marked as [Obsolete] so you can’t say you weren’t warned! If you are using these, then you need to stay on a 3.x release of the client.

Start

This hasn’t been around since well, 3.0 I think, and has certainly deprecated for a long time. As it wouldn’t work in 3.5 onwards anyway I’m content to remove it.

Create(string, params object[]) 4/n

Again, marked as [Obsolete] and you should be using the alternative Create options instead.

Return<T>(string, CypherResultMode)

This was accidently made public, it was never intended to be, and as it has had at least 1 major version of Obsolete-ness, it’s gone.

StartBit

This pretty much comes with the Start code above.

Gremlin support

If you’re using Gremlin – there are no benefits to this version of the client, so stick with what you have. You should progress to Cypher if you can, it’s actively developed and is very closely linked to the new GQL standards that ISO have started to work on.

From a ‘career’ point of view, this means that learning Cypher is like learning GQL but with maybe a different accent, but not a different language. As GQL becomes a standard, other Graph databases will start to use it, and you’ll be ahead of the game.

Finally

Some other bits of tidying up!

URIs

You may (or may not) have noticed that the URI for the client has changed from: https://github.com/Readify/Neo4jClient to https://github.com/DotNet4Neo4j/Neo4jClient. This means that the project is now part of an Organisation (DotNet4Neo4j) which is focused on things that link Neo4j and .NET together.

Err.

I think that’s about it.

༼ つ ◕_◕ ༽つ

Neo4jClient turns 3.0

Well, version wise anyhow!

This is a pretty big release and is one I’ve been working on for a while now (sorry!). Version 3.0 of the client finally adds support for Bolt. When Neo4j released version 3.0 of their database, they added a new binary protocol called Bolt designed to be faster and easier on the ol’ network traffic.

For all versions of Neo4jClient prior to 3.x you could only access the DB via the REST protocol (side effect – you also had to use the client to access any version of Neo4j prior to 3.x).

I’ve tried my hardest to minimise the disruption that could happen, and I’m pretty happy to have it down to mainly 1 line change (assuming you’re passing an IGraphClient around – you are right??)

So without further ado:

var client = new GraphClient(new Uri("http://localhost:7474/db/data"), "user", "pass");`

becomes:

var client = new BoltGraphClient(new Uri("bolt://localhost:7687"), "user", "pass");

I’ve gone through a lot of trials with various objects, and I know others have as well (thank you!) but I’m sure there will still be errors – nothing is bug free! So raise an issue – ask on StackOverflow or Twitter 🙂

If you’re using `PathResults` you will need to swap to `BoltPathResults` – the format changed dramatically between the REST version and the Bolt version – and there’s not a lot I can do about it I’m afraid!

Excel & Neo4j? Let’s code that! (VSTO edition)

So you have a new Graph Database, it’s looking snazzy and graphy and all that, but, well – you really want to see it in a tabular format, ‘cos you’ve got this Excel program hanging about, and well – who doesn’t love a bit of tabular data?

Obviously there are loads of reasons why you might want that data – maybe to do a swanky graph, perhaps to pass over to the boss. You can also get that data into Excel in a few ways –

  • Open up from the Web – you can try opening up the REST endpoint in Excel directly – (I say try because quite frankly – it’s not looking like a good option)
  • Create an application to export to CSV – this is easy – writing a CSV/TSV/#SV is a doddle (in any language) but does mean you have to give it to people to run, and that might give more headaches – however it’s an option!
  • Create an Excel Addin that runs within Excel – slightly more complicated as you need to interact with Excel directly – but does have the benefit that maybe you can use it to send data back to the db as well..

As you can imagine, this is about doing the third option – to be honest, I would only ever pick options 2 or 3, and if I’m super honest – I would normally go for option 2 – as it’s the simplest. Option 3 however has some benefits I’d like to explore.

If you want to look at the project – you can find it at: https://github.com/DotNet4Neo4j/Neo4jDriverExcelAddin

I’ll be using the official driver (Neo4j.Driver) and VSTO addins, with VS 2017.

Onwards!

Sidenote

As I was writing this, I was going to do my usual – step-by-step approach, so went to take a screenshot and noticed this:

image

So we’re going to do a quick overview of the VSTO version, then the next post will tuck into the Excel Web version which looks snazzier – but I don’t have an example as of yet…

Onwards again!

Sidenote 2: Sidenote Harder

As the code is on github I’m not going to show everything, merely the important stuff, as you can get all the code and check it out for yourself!

So – pick the new VSTO addin option:

image

And create your project. You’ll end up with something like this:

image

OK, so an addin needs a few things –

  1. A button on the ribbon
  2. A form (yes, WinForm) to get our input (cypher)
  3. The code that executes stuff

The Form

That’s right. Form. Actually – UserControl, but still WinForms (Hello 2000), let’s add our interface to the project, right click and ‘Add New Item’:

image

For those who’ve not had the pleasure before, the key thing to learn is how the Anchors work to prevent your form doing weird stuff when it’s resized.

Add a textbox to the control:

image

Single line eh? That’s not very useful – let’s MULTI-LINE!

Right–click on the box and select properties and that properties window you never use pops up, ready to be used! Change the name to something useful – or leave it  – it’s up to you – the key settings are Anchor and Multiline. Multline should be true, Anchor should then be all the anchors:

image

If you resize your whole control now, you should see that your textbox will expand and contract with it – good times!

Drag a button onto that form and place it to the bottom right of your textbox, and now we need to set the anchors again, but this time to Bottom, Right so it will move with resizing correctly – also we should probably change the Text to something more meaningful than button1 – again – don’t let me preach UX to you! Play around, make it bigger, change the colour, go WILD.

Once your button dreams have been realised – double click on the button to be taken to the code behind.First we’ll add some custom EventArgs:

internal class ExecuteCypherQueryArgs : EventArgs
{
     public string Cypher { get; set; }
}

and then a custom EventHandler:

internal EventHandler<ExecuteCypherQueryArgs> ExecuteCypher;

Then we call that event when the button is pressed, so the UserControl code looks like:

public partial class ExecuteQuery : UserControl
{
    internal EventHandler<ExecuteCypherQueryArgs> ExecuteCypher;

    public ExecuteQuery()
    {
        InitializeComponent();
    }

    private void _btnExecute_Click(object sender, EventArgs e)
    {
        if (string.IsNullOrWhiteSpace(_txtCypher.Text))
            return;

        ExecuteCypher?.Invoke(this, new ExecuteCypherQueryArgs { Cypher = _txtCypher.Text });
    }
}

The Ribbon

OK, we now have a form, but no way to see said form, so we need a Ribbon. Let’s add a new  Ribbon (XML) to our project

image

Open up the new .xml file and add the following to the &lt;group&gt; elements:

<button id="btnShowHide" label="Show/Hide" onAction="OnShowHideButton"/>

Now open the .cs file that has the same name as your .xml and add the following:

internal event EventHandler ShowHide;

public void OnShowHideButton(Office.IRibbonControl control)
{
    ShowHide?.Invoke(this, null);
}

Basically, we raise an event when the button is pressed. But what is listening for this most epic of notifications??? That’s right.. it’s:

ThisAddin.cs

The unfortunate part about going from here on in is that this is largely plumbing… ugh! The code around how to show/hide a form I’ll skip over – it’s all in the GitHub repo and you can read it easily enough.

There are a couple of bits of interest – one is the ThisAddin_Startup method, in which we create our Driver instance:

private void ThisAddIn_Startup(object sender, EventArgs e)
{
     _driver = GraphDatabase.Driver(new Uri("bolt://localhost"), AuthTokens.Basic("neo4j", "neo"));
}

To improve this, you’d want to get the URL and login details from the user somehow, perhaps a settings form – but I’ll leave that to you! – The important bit is that we store the IDriver instance in the addin. We only want one instance of a Driver per Excel, so this is fine.

The other interesting method is the ExecuteCypher method – (which is hooked up to in the InitializePane method) – This takes the results of our query and puts it into Excel:

private void ExecuteCypher(object sender, ExecuteCypherQueryArgs e)
{
    var worksheet = ((Worksheet) Application.ActiveSheet);

    using (var session = _driver.Session())
    {
        var result = session.Run(e.Cypher);
        int row = 1;
        
        foreach (var record in result)
        {
            var range = worksheet.Range[$"A{row++}"]; //TODO: Hard coded range
            range.Value2 = record["UserId"].As<string>(); //TODO: Hard coded 'UserId' here.
        }
     }
}

Again – HardCoded ranges and ‘Columns’ (UserId) – you’ll want to change these to make sense for your queries, or even better, just make them super generic.

Summing Up

So now we’re at this stage, we have an Excel addin using VSTO that can call Cypher and display the results, there are things we probably want to add – firstly – remove all the hard coded stuff. But what about being able to ‘update’ results based on your query?? That’d be cool – and maybe something we’ll look at in the next addin based post (on Web addins).