neo4j

This model serves as a demonstration of how fine-tuning foundational models using the Neo4j-Text2Cypher(2024) Dataset (link) can enhance performance on the Text2Cypher task.\ Please note, this is part of ongoing research and exploration, aimed at highlighting the dataset's potential rather than a production-ready solution. Base model: google/gemma-2-9b-it \ Dataset: neo4j/text2cypher-2024v1 An overview of the finetuned models and benchmarking results are shared at Link1 and Link2 Have ideas or insights? Contact us: Neo4j/Team-GenAI We need to be cautious about a few risks: In our evaluation setup, the training and test sets come from the same data distribution (sampled from a larger dataset). If the data distribution changes, the results may not follow the same pattern. The datasets used were gathered from publicly available sources. Over time, foundational models may access both the training and test sets, potentially achieving similar or even better results. 1 x A100 PCIe 31 vCPU 117 GB RAM runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04 On-Demand - Secure Cloud 60 GB Disk 60 GB Pod Volume loraconfig = LoraConfig( r=64, loraalpha=64, targetmodules=targetmodules, loradropout=0.05, bias="none", tasktype="CAUSALLM", ) sftconfig = SFTConfig( datasettextfield=datasettextfield, perdevicetrainbatchsize=4, gradientaccumulationsteps=8, datasetnumproc=16, maxseqlength=1600, loggingdir="./logs", numtrainepochs=1, learningrate=2e-5, savesteps=5, savetotallimit=1, loggingsteps=5, outputdir="outputs", optim="pagedadamw8bit", savestrategy="steps", ) bnbconfig = BitsAndBytesConfig( loadin4bit=True, bnb4bitusedoublequant=True, bnb4bitquanttype="nf4", bnb4bitcomputedtype=torch.bfloat16, ) Example Cypher generation cypher\nMATCH...cypher\nMATCH... NOTE on creating your own schemas: In the dataset we used, the schemas are already provided. They are created either by Directly using the schema the input data source provided OR Creating schema using neo4j-graphrag package (Check: SchemaReader.getschema(...) function) In your own Neo4j database, you can utilize `neo4j-graphrag package::SchemaReader` functions This model is a fine-tuned derivative of the Gemma base model. It is governed by the Gemma Terms of Use, including restrictions such as: - Non-commercial use only - No sublicensing - Compliance with applicable laws and regulations See the included `GEMMALICENSE` and `NOTICE` files for full details.