Skip to content

Model updates #1409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kaustubh-darekar wants to merge 5 commits into
base: dev
Choose a base branch
from
Draft

Model updates #1409

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
aafa79c
model updates, integration test script upgrade, embedding model upgrade
kaustubh-darekar Dec 3, 2025
f7bc1dc
model update
kaustubh-darekar Dec 4, 2025
768c481
running test sequentially
kaustubh-darekar Dec 4, 2025
00d2fd5
Updating llm models, embedding models
kaustubh-darekar Dec 5, 2025
b8aa905
Merge branch 'dev' of https://github.com/neo4j-labs/llm-graph-builder…
kaustubh-darekar Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Run the application using the default `docker-compose` configuration.
- By default, only OpenAI and Diffbot are enabled. Gemini requires additional GCP configurations.
- Use the `VITE_LLM_MODELS_PROD` variable to configure the models you need. Example:
```bash
VITE_LLM_MODELS_PROD="openai_gpt_4o,openai_gpt_4o_mini,diffbot,gemini_1.5_flash"
VITE_LLM_MODELS_PROD="openai_gpt_5.1,openai_gpt_5_mini,diffbot,gemini_2.5_flash"
```

2. **Input Sources**:
Expand Down Expand Up @@ -199,10 +199,10 @@ VITE_BACKEND_API_URL=${VITE_BACKEND_API_URL-backendurl}
| DUPLICATE_TEXT_DISTANCE | Mandatory | 5 | This value used to find distance for all node pairs in the graph and calculated based on node properties |
| DUPLICATE_SCORE_VALUE | Mandatory | 0.97 | Node score value to match duplicate node |
| EFFECTIVE_SEARCH_RATIO | Mandatory | 1 | |
| GRAPH_CLEANUP_MODEL | Optional | 0.97 | Model name to clean-up graph in post processing |
| GRAPH_CLEANUP_MODEL | Optional | "openai_gpt_5_mini" | Model name to clean-up graph in post processing |
| MAX_TOKEN_CHUNK_SIZE | Optional | 10000 | Maximum token size to process file content |
| YOUTUBE_TRANSCRIPT_PROXY| Optional | | Proxy key to process youtube video for getting transcript |
| EMBEDDING_MODEL | Optional | all-MiniLM-L6-v2 | Model for generating the text embedding (all-MiniLM-L6-v2 , openai , vertexai) |
| EMBEDDING_MODEL | Optional | | Model for generating the text embedding (default all-MiniLM-L6-v2 , openai , vertexai, titan) |
| IS_EMBEDDING | Optional | true | Flag to enable text embedding |
| KNN_MIN_SCORE | Optional | 0.94 | Minimum score for KNN algorithm |
| GEMINI_ENABLED | Optional | False | Flag to enable Gemini |
Expand All @@ -219,7 +219,7 @@ VITE_BACKEND_API_URL=${VITE_BACKEND_API_URL-backendurl}
| LANGCHAIN_ENDPOINT | Optional | https://api.smith.langchain.com | Endpoint for Langchain API |
| ENTITY_EMBEDDING | Optional | False | If set to True, It will add embeddings for each entity in database |
| LLM_MODEL_CONFIG_ollama_<model_name> | Optional | | Set ollama config as - model_name,model_local_url for local deployments |
| RAGAS_EMBEDDING_MODEL | Optional | openai | embedding model used by ragas evaluation framework |
| RAGAS_EMBEDDING_MODEL | Optional | | embedding model used by ragas evaluation framework |
| |
| **FRONTEND ENV**
| VITE_BLOOM_URL | Mandatory | https://workspace-preview.neo4j.io/workspace/explore?connectURL={CONNECT_URL}&search=Show+me+a+graph&featureGenAISuggestions=true&featureGenAISuggestionsInternal=true | URL for Bloom visualization |
Expand Down
44 changes: 20 additions & 24 deletions backend/example.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
OPENAI_API_KEY = "" #This is required if you are using openai embedding model
EMBEDDING_MODEL = "all-MiniLM-L6-v2" #this can be openai or vertexai or by default all-MiniLM-L6-v2
RAGAS_EMBEDDING_MODEL = "openai" #Keep blank if you want to use all-MiniLM-L6-v2 for ragas embeddings
EMBEDDING_MODEL = "" #values can be blank or "openai" or "vertexai" "titan" - defaults to all-MiniLM-L6-v2
RAGAS_EMBEDDING_MODEL = "" #values can be blank or "openai"
IS_EMBEDDING = "TRUE"
KNN_MIN_SCORE = "0.94"
# Enable Gemini (default is False) | Can be False or True
GEMINI_ENABLED = False
GEMINI_ENABLED = "False" #Keep true if you are using gemini model
# Enable Google Cloud logs (default is False) | Can be False or True
GCP_LOG_METRICS_ENABLED = False
NUMBER_OF_CHUNKS_TO_COMBINE = 6
Expand All @@ -19,7 +19,7 @@ LANGCHAIN_API_KEY = ""
LANGCHAIN_PROJECT = ""
LANGCHAIN_TRACING_V2 = ""
LANGCHAIN_ENDPOINT = ""
GCS_FILE_CACHE = "" #save the file into GCS or local, SHould be True or False
GCS_FILE_CACHE = "" #save the file into GCS or local, Should be True or False
NEO4J_USER_AGENT=""
ENABLE_USER_AGENT = ""
LLM_MODEL_CONFIG_model_version=""
Expand All @@ -28,30 +28,26 @@ DUPLICATE_SCORE_VALUE =0.97
DUPLICATE_TEXT_DISTANCE =3
DEFAULT_DIFFBOT_CHAT_MODEL="openai_gpt_4o" #whichever model specified here , need to add config for that model in below format)
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate model configuration for fireworks_gpt_oss. This model appears on both line 29 and line 48. Please remove the duplicate entry.

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DEFAULT_DIFFBOT_CHAT_MODEL is set to openai_gpt_4o, which is a model that was removed in this PR. This creates an inconsistency - the default references a model that may not be configured. Consider updating this to one of the new models like openai_gpt_5_mini or another valid model that's included in the updated model list.

Suggested change
DEFAULT_DIFFBOT_CHAT_MODEL="openai_gpt_4o" #whichever model specified here , need to add config for that model in below format)
DEFAULT_DIFFBOT_CHAT_MODEL="openai_gpt_5_mini" #whichever model specified here , need to add config for that model in below format)

Copilot uses AI. Check for mistakes.
#examples
LLM_MODEL_CONFIG_openai_gpt_3.5="gpt-3.5-turbo-0125,openai_api_key"
LLM_MODEL_CONFIG_openai_gpt_4o_mini="gpt-4o-mini-2024-07-18,openai_api_key"
LLM_MODEL_CONFIG_openai_gpt_4o="gpt-4o-2024-11-20,openai_api_key"
LLM_MODEL_CONFIG_openai_gpt_4.1_mini="gpt-4.1-mini,openai_api_key"
LLM_MODEL_CONFIG_openai_gpt_4.1="gpt-4.1,openai_api_key"
LLM_MODEL_CONFIG_openai_gpt_o3_mini="o3-mini-2025-01-31,openai_api_key"
LLM_MODEL_CONFIG_gemini_1.5_pro="gemini-1.5-pro-002"
LLM_MODEL_CONFIG_gemini_1.5_flash="gemini-1.5-flash-002"
LLM_MODEL_CONFIG_gemini_2.0_flash="gemini-2.0-flash-001"
LLM_MODEL_CONFIG_openai_gpt_5.1="gpt-5.1,openai-key"
LLM_MODEL_CONFIG_openai_gpt_5_mini="gpt-5-mini,openai-key"
LLM_MODEL_CONFIG_openai_gpt_4.1="gpt-4.1,openai-key"
LLM_MODEL_CONFIG_openai_gpt_4.1_mini="gpt-4.1-mini,openai-key"
LLM_MODEL_CONFIG_gemini_2.5_flash="gemini-2.5-flash"
LLM_MODEL_CONFIG_gemini_2.5_pro="gemini-2.5-pro"
LLM_MODEL_CONFIG_diffbot="diffbot,diffbot_api_key"
LLM_MODEL_CONFIG_azure_ai_gpt_35="azure_deployment_name,azure_endpoint or base_url,azure_api_key,api_version"
LLM_MODEL_CONFIG_groq_llama3.1_8b="llama-3.1-8b-instant,base_url,groq_api_key"
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent model naming: The model is referenced as anthropic_claude_4.5_sonnet in the constants but the example configuration uses a different naming pattern claude-sonnet-4-5-20250929. For maintainability, ensure the mapping between the constant name and the actual model identifier is clear and documented.

Suggested change
LLM_MODEL_CONFIG_groq_llama3.1_8b="llama-3.1-8b-instant,base_url,groq_api_key"
LLM_MODEL_CONFIG_groq_llama3.1_8b="llama-3.1-8b-instant,base_url,groq_api_key"
# Note: The constant name 'anthropic_claude_4.5_sonnet' maps to the model identifier 'claude-sonnet-4-5-20250929' as used by Anthropic.

Copilot uses AI. Check for mistakes.
LLM_MODEL_CONFIG_anthropic_claude_4.5_sonnet="claude-sonnet-4-5-20250929,anthropic_api_key"
LLM_MODEL_CONFIG_llama4_maverick="Llama-4-Maverick-17B-128E-Instruct-FP8,https://api.llama.com/compat/v1/,LLM|1207839134334841|NT9iYvy201sMmsOJB6spkslwoiM"
LLM_MODEL_CONFIG_azure_ai_gpt_4o="gpt-4o,https://YOUR-ENDPOINT.openai.azure.com/,azure_api_key,api_version"
LLM_MODEL_CONFIG_groq_llama3_70b="model_name,base_url,groq_api_key"
LLM_MODEL_CONFIG_anthropic_claude_4_sonnet="model_name,anthropic_api_key" #model_name="claude-sonnet-4-20250514"
LLM_MODEL_CONFIG_fireworks_llama4_maverick="model_name,fireworks_api_key"
LLM_MODEL_CONFIG_fireworks_qwen3_30b="accounts/fireworks/models/qwen3-30b-a3b,fireworks_api_key"
LLM_MODEL_CONFIG_fireworks_gpt_oss="accounts/fireworks/models/gpt-oss-120b,fireworks_api_key"
LLM_MODEL_CONFIG_fireworks_deepseek_v3="accounts/fireworks/models/deepseek-v3p1,fireworks_api_key"
LLM_MODEL_CONFIG_bedrock_nova_micro_v1="amazon.nova-micro-v1:0,aws_access_key,aws_secret_key,region_name"
LLM_MODEL_CONFIG_bedrock_nova_lite_v1="amazon.nova-lite-v1:0,aws_access_key,aws_secret_key,region_name"
LLM_MODEL_CONFIG_bedrock_nova_pro_v1="amazon.nova-pro-v1:0,aws_access_key,aws_secret_key,region_name"
LLM_MODEL_CONFIG_ollama_llama3="llama3_model_name,model_local_url"
YOUTUBE_TRANSCRIPT_PROXY="https://user:pass@domain:port"
EFFECTIVE_SEARCH_RATIO=5
GRAPH_CLEANUP_MODEL="openai_gpt_4o"
BEDROCK_EMBEDDING_MODEL="model_name,aws_access_key,aws_secret_key,region_name" #model_name="amazon.titan-embed-text-v1"
LLM_MODEL_CONFIG_bedrock_nova_micro_v1="model_name,aws_access_key,aws_secret_key,region_name" #model_name="amazon.nova-micro-v1:0"
LLM_MODEL_CONFIG_bedrock_nova_lite_v1="model_name,aws_access_key,aws_secret_key,region_name" #model_name="amazon.nova-lite-v1:0"
LLM_MODEL_CONFIG_bedrock_nova_pro_v1="model_name,aws_access_key,aws_secret_key,region_name" #model_name="amazon.nova-pro-v1:0"
LLM_MODEL_CONFIG_fireworks_deepseek_r1="model_name,fireworks_api_key" #model_name="accounts/fireworks/models/deepseek-r1"
LLM_MODEL_CONFIG_fireworks_deepseek_v3="model_name,fireworks_api_key" #model_name="accounts/fireworks/models/deepseek-v3"
GRAPH_CLEANUP_MODEL="openai_gpt_5_mini"
BEDROCK_EMBEDDING_MODEL="model_name,aws_access_key,aws_secret_key,region_name" #model_name="amazon.titan-embed-text-v2.0"
MAX_TOKEN_CHUNK_SIZE=2000 #Max token used to process/extract the file content.
6 changes: 3 additions & 3 deletions backend/score.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,9 @@ async def __call__(self, scope: Scope, receive: Receive, send: Send):
)
app.add_middleware(SessionMiddleware, secret_key=os.urandom(24))

is_gemini_enabled = os.environ.get("GEMINI_ENABLED", "False").lower() in ("true", "1", "yes")
if is_gemini_enabled:
add_routes(app,ChatVertexAI(), path="/vertexai")
# is_gemini_enabled = os.environ.get("GEMINI_ENABLED", "False").lower() in ("true", "1", "yes")
# if is_gemini_enabled:
# add_routes(app,ChatVertexAI(), path="/vertexai")
Comment on lines +115 to +117
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment appears to contain commented-out code.

Suggested change
# is_gemini_enabled = os.environ.get("GEMINI_ENABLED", "False").lower() in ("true", "1", "yes")
# if is_gemini_enabled:
# add_routes(app,ChatVertexAI(), path="/vertexai")
is_gemini_enabled = os.environ.get("GEMINI_ENABLED", "False").lower() in ("true", "1", "yes")
if is_gemini_enabled:
add_routes(app, ChatVertexAI(), path="/vertexai")

Copilot uses AI. Check for mistakes.
Comment on lines +115 to +117
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment appears to contain commented-out code.

Suggested change
# is_gemini_enabled = os.environ.get("GEMINI_ENABLED", "False").lower() in ("true", "1", "yes")
# if is_gemini_enabled:
# add_routes(app,ChatVertexAI(), path="/vertexai")

Copilot uses AI. Check for mistakes.

app.add_api_route("/health", health([healthy_condition, healthy]))

Expand Down
2 changes: 1 addition & 1 deletion backend/src/communities.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
MAX_WORKERS = 10
MAX_COMMUNITY_LEVELS = 3
MIN_COMMUNITY_SIZE = 1
COMMUNITY_CREATION_DEFAULT_MODEL = "openai_gpt_4o"
COMMUNITY_CREATION_DEFAULT_MODEL = "openai_gpt_4.1"


CREATE_COMMUNITY_GRAPH_PROJECTION = """
Expand Down
1 change: 0 additions & 1 deletion backend/src/graphDB_dataAccess.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,6 @@ def connection_check_and_get_vector_dimensions(self,database):

embedding_model = os.getenv('EMBEDDING_MODEL')
embeddings, application_dimension = load_embedding_model(embedding_model)
logging.info(f'embedding model:{embeddings} and dimesion:{application_dimension}')

gds_status = self.check_gds_version()
write_access = self.check_account_access(database=database)
Expand Down
7 changes: 2 additions & 5 deletions backend/src/llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def get_llm(model: str):
)
elif "openai" in model:
model_name, api_key = env_value.split(",")
if "o3-mini" in model:
if "mini" in model:
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition if "mini" in model is too broad and may incorrectly match models that contain "mini" anywhere in their name (e.g., "mini-processor", "gemini"). This could lead to incorrect model initialization. Consider using a more specific pattern like if "o3-mini" in model or "gpt-4.1-mini" in model or "gpt-5-mini" in model or another precise matching strategy.

Suggested change
if "mini" in model:
# Only match specific "mini" models to avoid accidental matches (e.g., "gemini")
if re.search(r'(^|\W)(o3-mini|gpt-4\.1-mini|gpt-5-mini)($|\W)', model):

Copilot uses AI. Check for mistakes.
llm= ChatOpenAI(
api_key=api_key,
model=model_name)
Expand Down Expand Up @@ -189,17 +189,14 @@ async def get_graph_document_list(
else:
node_properties = ["description"]
relationship_properties = ["description"]
TOOL_SUPPORTED_MODELS = {"qwen3", "deepseek"}
model_name = get_llm_model_name(llm)
ignore_tool_usage = not any(pattern in model_name for pattern in TOOL_SUPPORTED_MODELS)
logging.info(f"Keeping ignore tool usage parameter as {ignore_tool_usage}")
llm_transformer = LLMGraphTransformer(
llm=llm,
node_properties=node_properties,
relationship_properties=relationship_properties,
allowed_nodes=allowedNodes,
allowed_relationships=allowedRelationship,
ignore_tool_usage=ignore_tool_usage,
ignore_tool_usage=True,
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoding ignore_tool_usage=True for all models removes the previous conditional logic that checked for tool-supported models. This may negatively impact performance or functionality for models that do support tool usage (like qwen3 and deepseek). Please verify that ignoring tool usage for all models is the intended behavior.

Copilot uses AI. Check for mistakes.
additional_instructions=ADDITIONAL_INSTRUCTIONS+ (additional_instructions if additional_instructions else "")
)

Expand Down
2 changes: 1 addition & 1 deletion backend/src/post_processing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ def graph_schema_consolidation(graph):
messages=[("system", GRAPH_CLEANUP_PROMPT), ("human", "{input}")],
partial_variables={"format_instructions": parser.get_format_instructions()}
)
graph_cleanup_model = os.getenv("GRAPH_CLEANUP_MODEL", 'openai_gpt_4o')
graph_cleanup_model = os.getenv("GRAPH_CLEANUP_MODEL")
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the default value from GRAPH_CLEANUP_MODEL environment variable can cause issues if the variable is not set. The code at line 208 calls get_llm(graph_cleanup_model) which will fail if graph_cleanup_model is None. Consider keeping a sensible default value or adding validation to handle the None case.

Suggested change
graph_cleanup_model = os.getenv("GRAPH_CLEANUP_MODEL")
graph_cleanup_model = os.getenv("GRAPH_CLEANUP_MODEL", "gpt-3.5-turbo")

Copilot uses AI. Check for mistakes.
llm, _ = get_llm(graph_cleanup_model)
chain = prompt | llm | parser

Expand Down
6 changes: 3 additions & 3 deletions backend/src/shared/common_fn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,13 +113,13 @@ def load_embedding_model(embedding_model_name: str):
logging.info(f"Embedding: Using OpenAI Embeddings , Dimension:{dimension}")
elif embedding_model_name == "vertexai":
embeddings = VertexAIEmbeddings(
model="textembedding-gecko@003"
model="gemini-embedding-001"
)
dimension = 768
dimension = 3072
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Vertex AI embedding model is updated to gemini-embedding-001 with dimension 3072. However, this dimension appears incorrect. According to Google's documentation, the text-embedding-004 model has 768 dimensions, and gemini-embedding models typically have 768 dimensions, not 3072. Please verify the correct dimension for this embedding model.

Suggested change
dimension = 3072
dimension = 768

Copilot uses AI. Check for mistakes.
logging.info(f"Embedding: Using Vertex AI Embeddings , Dimension:{dimension}")
elif embedding_model_name == "titan":
embeddings = get_bedrock_embeddings()
dimension = 1536
dimension = 1024
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Bedrock Titan embedding dimension is updated to 1024, but this should be verified. Amazon Titan Embed Text v1 uses 1536 dimensions, while v2 can use 256, 512, or 1024 dimensions depending on configuration. Please ensure this dimension matches the actual Bedrock embedding model being used (as referenced in the BEDROCK_EMBEDDING_MODEL configuration).

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The embedding dimension for the Bedrock Titan model is changed from 1536 to 1024. However, Amazon Titan Text Embeddings v1 produces 1536-dimensional embeddings, while v2 produces 1024-dimensional embeddings. The comment on line 52 of example.env mentions amazon.titan-embed-text-v2.0, which would have 1024 dimensions. Please ensure this dimension matches the actual Titan model version being used by the get_bedrock_embeddings() function.

Copilot uses AI. Check for mistakes.
logging.info(f"Embedding: Using bedrock titan Embeddings , Dimension:{dimension}")
else:
# embeddings = HuggingFaceEmbeddings(model_name="./local_model")
Expand Down
Loading
Loading