Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions .github/steps/1-genai-prototype.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ With GitHub Models, you can use GitHub Personal Access Tokens (PAT) to authentic
1. Right click [GitHub Models](https://github.com/marketplace/models) and open in a new tab

> [!NOTE]
> Open the link IN A NEW TAB so you can keep this page open for reference.
> Open the link IN A NEW TAB so you can keep this page open for reference. You can use the _Split Screen_ feature on Edge to have this instructions and GitHub Models/ VS Code open side-by-side

2. Click on **explore the full model catalog** to see the available models.

Expand Down Expand Up @@ -142,7 +142,7 @@ Now that you have a better understanding of the models from the GitHub Models pl

## 📌 Exercise: Convert a hand-drawn sketch to a web page

1. Download the contoso website hand-drawn sketch from [this link](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/contoso_layout_sketch.jpg), and save it `contoso_layout_sketch.jpg` in the same directory as your `sample.js` file.
1. Download the contoso website hand-drawn sketch from [this link](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/contoso_layout_sketch.jpg), _(right click and open in new tab)_. and save it `contoso_layout_sketch.jpg` in the same directory as your `sample.js` file.

> **Note**
>
Expand Down Expand Up @@ -214,11 +214,12 @@ To complete this quest and **AUTOMATICALLY UPDATE** your progress, you MUST push

Here are some additional resources to help you learn more about experimenting with AI models and building prototypes:

- [About GitHub Models](https://docs.github.com/en/github-models/about-github-models)
- [Choosing the right AI model for your task](https://docs.github.com/en/copilot/using-github-copilot/ai-models/choosing-the-right-ai-model-for-your-task)
- [Introducing GitHub Models: A new generation of AI engineers building on GitHub](https://github.blog/news-insights/product-news/introducing-github-models/)
- [📹 DEM500: Prototype, build, and deploy AI apps quickly with GitHub Models](https://build.microsoft.com/en-US/sessions/DEM500?source=sessions)
- [What is the AI Toolkit for Visual Studio Code?](https://learn.microsoft.com/en-us/windows/ai/toolkit/)
- [Lesson 1: Introduction to Generative AI and LLMs for JavaScript Developers](https://github.com/microsoft/generative-ai-with-javascript/tree/main/lessons/01-intro-to-genai#lesson-1-introduction-to-generative-ai-and-llms-for-javascript-developers)
- [Lesson 2: Writing your first AI app](https://github.com/microsoft/generative-ai-with-javascript/blob/main/lessons/02-first-ai-app/README.md#lesson-2-writing-your-first-ai-app)
- [Lesson 3: Prompt Engineering](https://github.com/microsoft/generative-ai-with-javascript/blob/main/lessons/03-prompt-engineering/README.md)
- [GitHub Models - Your AI toolbox built into GitHub](https://github.com/features/models)
- [Install the AI Toolkit](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio&ssr=false#overview) to browse and download public and custom models; author, test and evaluate prompts; fine-tune; and use them in your applications.
- [What is the AI Toolkit for Visual Studio Code?](https://learn.microsoft.com/en-us/windows/ai/toolkit/)
- [Build Apps and Agents with Visual Studio Code and Azure blog](https://devblogs.microsoft.com/blog/build-apps-and-agents-with-visual-studio-code-and-azure)
- A curated set of essential extensions for building generative AI applications and agents in Visual Studio Code. Install [the Microsoft AI Tools Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.microsoft-ai-tools-pack)
- [Microsoft AI Tools Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.microsoft-ai-tools-pack), a curated set of essential extensions for building generative AI applications and agents in VS Code
7 changes: 2 additions & 5 deletions .github/steps/2-to-azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ To continue working with your deployed model in VS Code, you will need to instal
6. Create a `.env` file in your project folder and update it with the following environment variables:

```bash
AZURE_INFERENCE_API_KEY=your_api_key
AZURE_INFERENCE_SDK_KEY=your_api_key
AZURE_INFERENCE_SDK_ENDPOINT=your_api_endpoint
```

Expand Down Expand Up @@ -126,8 +126,5 @@ Here are some additional resources to help you learn more about getting started

- [Azure AI Foundry Extension for Visual Studio Code is Available in Preview](https://devblogs.microsoft.com/foundry/azure-ai-foundry-vscode-extension-preview/)
- [Work with the Azure AI Foundry for Visual Studio Code extension docs](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/get-started-projects-vs-code)
- [📹 DEM500: Prototype, build, and deploy AI apps quickly with GitHub Models](https://build.microsoft.com/en-US/sessions/DEM500?source=sessions)
- [📹 BRK107: Develop, Build and Deploy LLM Apps using GitHub Models and Azure AI Foundry](https://build.microsoft.com/en-US/sessions/BRK107?source=sessions)
- [Build with the AI Foundry JavaScript SDK](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/sdk-overview?pivots=programming-language-javascript)
<!-- - [JavaScript SDK for Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/sdk-overview?tabs=sync&pivots=programming-language-python) -->
- A curated set of essential extensions for building generative AI applications and agents in Visual Studio Code. Install [the Microsoft AI Tools Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.microsoft-ai-tools-pack)
- [Microsoft AI Tools Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.microsoft-ai-tools-pack), a curated set of essential extensions for building generative AI applications and agents in VS Code
16 changes: 11 additions & 5 deletions .github/steps/3-add-chat-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,11 @@ In your current working directory, _(at the root)_, run the following command to
azd init -t Azure-Samples/vite-chat-interface
```

> [!Note]
> After running the above command, select **Keep my existing files unchanged** for the following option to prevent your README from being overwritten
>
> ![azd files option to kep](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/azd-files-options.png?raw=true)

This will initialize a new Vite project and add the necessary files and folders to your project:

```markdown
Expand Down Expand Up @@ -226,7 +231,7 @@ The project is already configured to deploy the webapp (frontend) to Azure Stati

```yaml
webapp:
project: packages/webapp
project: webapp
host: staticwebapp
language: js
dist: dist
Expand Down Expand Up @@ -254,11 +259,13 @@ module webapp 'br/public:avm/res/web/static-site:0.7.0' = {
}
}
```
However, remember you updated the path to the `webapp` folder when we moved it to the `packages` folder. **So change the project path in `azure.yaml` to `project: packages/webapp` for the webapp service**

The webapi service is not yet configured in the `azure.yaml` file. To add the webapi service, add the following code to the `azure.yaml` file inside the `services` node:

```yaml
webapi:
project: packages/webapp
project: packages/webapi
host: appservice
language: js
```
Expand All @@ -281,6 +288,7 @@ module webapi 'br/public:avm/res/web/site:0.15.1' = {
params: {
kind: 'app'
name: webapiName
tags: union(tags, { 'azd-service-name': webapiName })
serverFarmResourceId: serverfarm.outputs.resourceId
}
}
Expand All @@ -289,7 +297,7 @@ module webapi 'br/public:avm/res/web/site:0.15.1' = {
Declare the following parameters at the top of the `main.bicep` file to pass the names of the webapi and app service plan to the module:

```bash
param webapiName string = 'webapi'
param webapiName string = '<your-unique-string>' #use a unique string. avoid common names like webapi, website etc.
param appServicePlanName string = 'appserviceplan'
```

Expand Down Expand Up @@ -337,8 +345,6 @@ To complete this quest and **AUTOMATICALLY UPDATE** your progress, you MUST push

Here are some additional resources to help you learn more about tools used in this step:

- https://vite.dev/
- https://lit.dev/
- [Accelerate your journey to the cloud with azd](https://azure.github.io/awesome-azd/getting-started)
- [📹 BRK118: Accelerate Azure Development with GitHub Copilot, VS Code & AI](https://build.microsoft.com/en-US/sessions/BRK118?source=sessions)
- [Introducing the Azure Developer CLI (azd): A faster way to build apps for the cloud blog](https://devblogs.microsoft.com/azure-sdk/introducing-the-azure-developer-cli-a-faster-way-to-build-apps-for-the-cloud/)
Expand Down
22 changes: 12 additions & 10 deletions .github/steps/4-add-rag.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ In this step, you will learn how to add RAG (**R**etrieval-**A**ugmented **G**en

To complete this step, you will need to get a sample dataset in any format (e.g., PDF, CSV, JSON) to work with.

An example, will use a [sample Contoso Electronics Employee Handbook PDF](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/employee_handbook.pdf) file. **You can bring any file of your choice**, but make sure it contains relevant information that you want your AI app to use for RAG. The code provided here will work with any text-based file.
For this quest, as an example, will use a [sample Contoso Electronics Employee Handbook PDF](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/employee_handbook.pdf) file. **You can bring any file of your choice**, but make sure it contains relevant information that you want your AI app to use for RAG. The code provided here will work with any text-based file.

- Create a new folder `data` in the root of your project and move the file in it. To search and read your PDF, you will need to extract the text from it. You can use any PDF parser library of your choice, but for this example, we will use the `pdf-parse` library.

Expand All @@ -51,8 +51,7 @@ Open your server code `webapi/server.js` and modify it to include the following

```javascript
// add at the top of the file -----------------------------------------
import ModelClient, { isUnexpected } from "@azure-rest/ai-inference";
import { AzureKeyCredential } from "@azure/core-auth";
import { isUnexpected } from "@azure-rest/ai-inference";
import fs from "fs";
import path from "path";
import { fileURLToPath } from 'url';
Expand All @@ -79,7 +78,7 @@ async function loadPDF() {

const dataBuffer = fs.readFileSync(pdfPath);
const data = await pdfParse(dataBuffer);
pdfText = data.text; /
pdfText = data.text;
let currentChunk = "";
const words = pdfText.split(/\s+/);

Expand Down Expand Up @@ -135,7 +134,7 @@ app.post("/chat", async (req, res) => {
Use ONLY the following information from the handbook to answer the user's question.
If you can't find relevant information in the provided context, say so clearly.
--- EMPLOYEE HANDBOOK EXCERPTS ---
${sources.join('\n\n')}
${sources.join('')}
--- END OF EXCERPTS ---`
});
} else {
Expand Down Expand Up @@ -172,10 +171,9 @@ app.post("/chat", async (req, res) => {
}
});

const PORT = process.env.PORT || 3001;
app.listen(PORT, () => {
console.log(`AI API server running on port ${PORT}`);
});
// --------------------------------------------------------------------
// rest of the code from const PORT ...

```
</details>

Expand Down Expand Up @@ -415,4 +413,8 @@ To complete this quest and **AUTOMATICALLY UPDATE** your progress, you MUST push
## 📚 Further Reading

Here are some additional resources to help you learn more about RAG and how to implement it in your applications:
- [Lesson 5: Talk to your data with Retrieval-Augmented Generation (RAG)](https://github.com/microsoft/generative-ai-with-javascript/blob/main/lessons/05-rag/README.md)
- [Lesson 5: Talk to your data with Retrieval-Augmented Generation (RAG)](https://github.com/microsoft/generative-ai-with-javascript/blob/main/lessons/05-rag/README.md)
- [What is retrieval-augmented generation (RAG)?](https://azure.microsoft.com/en-us/resources/cloud-computing-dictionary/what-is-retrieval-augmented-generation-rag)
- [Retrieval augmented generation and indexes](https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/retrieval-augmented-generation)
- [Get started with Serverless AI Chat with RAG using LlamaIndex](https://learn.microsoft.com/en-us/azure/developer/javascript/ai/get-started-app-chat-template-llamaindex?tabs=github-codespaces)
- [Get started with Serverless AI Chat with RAG using LangChain.js](https://learn.microsoft.com/en-us/azure/developer/javascript/ai/get-started-app-chat-template-langchainjs?tabs=github-codespaces)
15 changes: 10 additions & 5 deletions .github/steps/5-frameworks.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ The current api code uses the Azure REST SDK directly. By switching to LangChain
Open `server.js` and replace:

```javascript
import ModelClient, { isUnexpected } from "@azure-rest/ai-inference";
import { AzureKeyCredential } from "@azure/core-auth";
import { isUnexpected } from "@azure-rest/ai-inference";
```

with:
Expand Down Expand Up @@ -71,6 +71,9 @@ const chatModel = new AzureChatOpenAI({
});
```

> [!Note]
> Update your `.env` with the missing variables

### Update chat endpoint

Replace the Azure REST SDK api call logic in the try-catch block (_app.post("/chat")_) with the following code:
Expand Down Expand Up @@ -124,7 +127,7 @@ import { ChatMessageHistory } from "langchain/stores/message/in_memory";
Store session histories, allowing you to maintain separate chat histories for different users or sessions.

```javascript
const sessionHistories = {};
const sessionMemories = {};
```

### Add a helper function to get/create a session history
Expand Down Expand Up @@ -170,8 +173,7 @@ app.post("/chat", async (req, res) => {
? {
role: "system",
content: sources.length > 0
? `You are a helpful assistant for Contoso Electronics. You must ONLY use the information provided below to answer.\n\n--- EMPLOYEE HANDBOOK EXCERPTS ---\n${sources.join('\n\n')}\n--- END OF EXCERPTS ---`
: `You are a helpful assistant for Contoso Electronics. The excerpts do not contain relevant information for this question. Reply politely: "I'm sorry, I don't know. The employee handbook does not contain information about that."`,
? `You are a helpful assistant for Contoso Electronics. You must ONLY use the information provided below to answer.\\n\\n--- EMPLOYEE HANDBOOK EXCERPTS ---\\n${sources.join('\\n\\n')}\\n--- END OF EXCERPTS ---`\n : `You are a helpful assistant for Contoso Electronics. The excerpts do not contain relevant information for this question. Reply politely: \"I'm sorry, I don't know. The employee handbook does not contain information about that.\"`,
}
: {
role: "system",
Expand Down Expand Up @@ -233,5 +235,8 @@ To complete this quest and **AUTOMATICALLY UPDATE** your progress, you MUST push
## 📚 Further Reading

Here are some additional resources to help you learn more about LangChain.js and its features:
- [Serverless AI Chat with RAG using LangChain.js sample](https://github.com/Azure-Samples/serverless-chat-langchainjs)
- [Get started with Serverless AI Chat with RAG using LangChain.js](https://github.com/Azure-Samples/serverless-chat-langchainjs)
- [LangChain.js x Microsoft docs](https://js.langchain.com/docs/integrations/platforms/microsoft/)
- [Ask YouTube: LangChain.js + Azure Quickstart](https://github.com/Azure-Samples/langchainjs-quickstart-demo)
- [LangChain.js + Azure: A Generative AI App Journey](https://techcommunity.microsoft.com/blog/educatordeveloperblog/langchain-js--azure-a-generative-ai-app-journey/4101258)
- [LangChain.js docs](https://js.langchain.com/docs/introduction/)
15 changes: 8 additions & 7 deletions .github/steps/6-build-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,16 +152,16 @@ options:

Save the `bing.yaml` file.

On the Agent Designer, click on the + icon next to the "Tools" section. This will prompt you to select a yaml file for the tool. Select the `bing.yaml` file you created earlier. Click on **Deploy to Azure AI Foundry** to deploy your updated agent with the new tool to Azure AI Foundry.
On the Agent Designer, click on the + icon next to the "Tools" section. This will prompt you to select a yaml file for the tool. Select the `bing.yaml` file you created earlier. Click on **Update Agent on Azure AI Foundry** to update your agent with the new tool to Azure AI Foundry.

![Add bing tool via extension](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/add-bing.png?raw=true)
![Add bing tool via extension](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/update-tool.png?raw=true)

Now that we have added the Bing Search API tool to our agent, we can test it in the Playground. Open the "Agent Playground" and send the agent a message like _"What's the weather in Nairobi right now?"_ The agent should use the Bing Search API tool to fetch the current weather information and respond with a friendly message.

![Weather with Bing](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/weather-with-bing.png?raw=true)


## Step 4️⃣: Agent playground to Code
<!-- ## Step 4️⃣: Agent playground to Code

The Agent Playground is a great way to test your agent's capabilities, but it's not the only way to interact with it. In this step, we will update our application to use the agent we just created. This will allow us to use the agent in our application and make it more useful.

Expand Down Expand Up @@ -463,10 +463,10 @@ On the app, select the **Agent** mode from the dropdown. Type a message in the i

If you ask the agent a question that requires real-time information, such as _"What's the current weather in Spain?"_, the agent should ground its response using the Bing Search API and provide you with the latest information.

![Weather in Spain in Agent mode](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/weather-in-spain.png?raw=true)
![Weather in Spain in Agent mode](https://github.com/Azure-Samples/JS-AI-Build-a-thon/blob/assets/jsai-buildathon-assets/weather-in-spain.png?raw=true) -->


## ✅ Activity: Push your updated code to the repository
## ✅ Activity: Push your updated code to the repository - TBD

### Quest Checklist

Expand Down Expand Up @@ -494,5 +494,6 @@ To complete this quest and **AUTOMATICALLY UPDATE** your progress, you MUST push
Here are some additional resources to help you learn more about building AI agents and extending their capabilities with tools:
- [Azure AI Agents JavaScript examples](https://github.com/Azure-Samples/azure-ai-agents-javascript)
- [Your First AI Agent in JS with Azure AI Agent Service](https://www.youtube.com/live/RNphlRKvmJQ?si=I3rUp-LmnvS008ym)
- [Create a new Agent - JS/TS](https://learn.microsoft.com/en-us/azure/ai-services/agents/quickstart?pivots=programming-language-javascript)
- [📹 DEMFP781: From Prompt to Product: Build an AI Agent That Generates UI](https://build.microsoft.com/en-US/sessions/DEMFP781?source=sessions)
- [Build Apps and Agents with Visual Studio Code and Azure blog](https://devblogs.microsoft.com/blog/build-apps-and-agents-with-visual-studio-code-and-azure)
- [📹 DEMFP781: From Prompt to Product: Build an AI Agent That Generates UI](https://build.microsoft.com/en-US/sessions/DEMFP781?source=sessions)
- [Build with the AI Foundry JavaScript SDK](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/sdk-overview?pivots=programming-language-javascript)
Loading