Dask with Neo4j

Dask DataFrame
1

Hello community,

What I am actually trying to achieve is to write some records on a remote neo4j database, however, it’s giving me the error 'TypeError("cannot pickle \'_thread.RLock\' object")'

At first I thought it could be connection object issue and I have to make each worker implement its own connection object separately by doing the following:

    def connect_worker_db():
        creds = getNeo4jCredentials()
        worker = get_worker()
        worker.driver = GraphDatabase.driver(creds.get('host'), auth=(creds.get('user'), creds.get('password')))


    client.register_worker_callbacks(connect_worker_db)

    def experiment():
        driver = get_worker().driver
        with driver.session() as session:
            result.apply(lambda activityName: addActivity(session, activityName),
                         meta=('activityNameEN', 'object')).compute()
        session.close()
        driver.close()

    result = client.submit(experiment)
    result.result()

But still giving me the same error. I am actually stuck at this point and I am trying to figure out what could be the solution for this, but so far I found there’s a serializableLock but I am not sure how this can fit in my use case or even how to use its APIs

Would be very grateful for your help