-
Notifications
You must be signed in to change notification settings - Fork 0
feat: Create server and bot #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
180c458
feat: Create server and bot
Kubaryt 68a4e3e
feat: Write unit, integration and e2e tests
Kubaryt 83a8fbe
feat: Create files for deployment
Kubaryt acaa8ae
docs: Create `README.md` and `.env.example` files
Kubaryt 76db50c
refactor: Handle missing user_id and remove code duplication from cre…
Kubaryt 4fb37fc
refactor: Return post if found via fetching active or archived threads
Kubaryt b6aac0a
feat: Add CI checks
Kubaryt 5d3991f
refactor: Split utils
Kubaryt 5e6c4b6
refactor: Change main branch to `master`
Kubaryt fa78dad
chore(deps): Update all dependencies and python to version 3.14
Kubaryt 7d9b118
refactor: Move github id to discord id mapping file to volume
Kubaryt 89e5e1f
fix: Make Docker service actually stop on SIGTERM
Kubaryt ebbba3e
chore: Disable pre-commit changing files during commiting
Kubaryt 59bce28
refactor: Make Dockerfile suited for production deployment
Kubaryt 9aacd60
chore: Add dependency groups and don't use dev dependencies on prod
Kubaryt 61ff65f
fix: Expect TypeError while fetching Github graphql API
Kubaryt 5c76c53
feat: Add error handling in bot task
Kubaryt 1fd7e69
fix: Use channel fetched after creating new tag, so tags aren't remov…
Kubaryt 1c9270b
fix: Remove trailing .> in task creation message
Kubaryt 017cf17
refactor: Better logging
Kubaryt 10f38af
refactor: Load shelve db paths from env
Kubaryt 1acb7de
chore: Regenerate lockfile and add lockfile tests
Kubaryt 29ae396
refactor: Move update_queue declaration to lifespan event
Kubaryt 4c2e6aa
refactor: Use Pydantic for request validation
Kubaryt 43afcc7
refactor: Merge simple_project_item_from_action_type with __init__ me…
Kubaryt eef746c
refactor: Merge `single_select_type_from_field_name` with `__init__` …
Kubaryt 45e7574
refactor: Move process_edition logic into interface method for Projec…
Kubaryt f2188ff
refactor: Move bot starting and bot exception handling logic to main.py
Kubaryt 7fb0978
refactor: Move logic out of `src/utils/__init__.py`
Kubaryt b1c060d
refactor: Remove `add_bot_log_prefix` function and add proper filter
Kubaryt 464d1f5
refactor: Enforce single responsibility principle and move signature …
Kubaryt cc28524
refactor: Tests overhaul
Kubaryt db280c5
chore: Reformat blank lines
Kubaryt 93913cd
refactor: Map Github item node_id to Discord post id
Kubaryt 1e75903
chore: Remove unused import
Kubaryt bd21e9b
refactor: Proper enum instance creation
Kubaryt 4457c8f
fix: Remove string matching when we use enum
Kubaryt ff9bb6f
chore: Add comments informing about reason for skipping ruff checks i…
Kubaryt e9da762
refactor: Remove code duplication from utils.github_api
Kubaryt eff874c
refactor: Use global logger variable instead of passing it each time
Kubaryt c6a2789
feat: Append link to GitHub project card in Discord post
Kubaryt 0e66bb6
refactor: Spawn task for process_update function in bot.py
Kubaryt b9e87e7
chore: Rename get_post_id to get_post_id_or_post
Kubaryt dc9ac12
refactor: Use `conftest.py` for defining fixtures
Kubaryt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| DISCORD_BOT_TOKEN=your-discord-bot-token | ||
| FORUM_CHANNEL_ID=your-forum-channel-id | ||
| GITHUB_TOKEN=your-github-token | ||
| GITHUB_PROJECT_NODE_ID=your-github-project-node-id | ||
| GITHUB_PROJECT_NUMBER=your-github-project-number | ||
| GITHUB_ORGANIZATION_NAME=your-github-organization-name | ||
| GITHUB_ID_TO_DISCORD_ID_MAPPING_PATH=path-to-github-username-to-discord-id-mapping.json | ||
| IP_ADDRESS=0.0.0.0 | ||
| PORT=8000 | ||
| GITHUB_WEBHOOK_SECRET=your-github-webhook-secret | ||
| # Shelve database | ||
| POST_ID_DB_PATH=path-to-post-id.db |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| name: CI | ||
| on: | ||
| push: | ||
| branches: [master] | ||
| pull_request: | ||
| branches: [master] | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| tests: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v7 | ||
|
|
||
| - name: Set up Python | ||
| run: uv python install | ||
|
|
||
| - name: Install all dependencies | ||
| run: uv sync --all-groups --locked | ||
|
|
||
| - name: Run Tests | ||
| run: uv run pytest | ||
|
|
||
| - name: Run Lint | ||
| run: uv run ruff check | ||
|
|
||
| - name: Run Formatter | ||
| run: uv run ruff format --check |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -205,3 +205,8 @@ cython_debug/ | |
| marimo/_static/ | ||
| marimo/_lsp/ | ||
| __marimo__/ | ||
|
|
||
| .idea/ | ||
|
|
||
| # Shelves database | ||
| post_id.db | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| repos: | ||
| # Run lint | ||
| - repo: https://github.com/astral-sh/ruff-pre-commit | ||
| # Ruff version. | ||
| rev: v0.14.5 | ||
| hooks: | ||
| # Run the linter. | ||
| - id: ruff | ||
| # Run the formatter. | ||
| - id: ruff-format | ||
| args: ["--check"] | ||
| # Run the tests. | ||
| - repo: local | ||
| hooks: | ||
| - id: pytest | ||
| name: pytest | ||
| entry: ./.venv/bin/pytest | ||
| language: system | ||
| types: [python] | ||
| pass_filenames: false | ||
| always_run: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| FROM python:3.14-slim-trixie | ||
| COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ | ||
|
|
||
| ADD . /app | ||
|
|
||
| WORKDIR /app | ||
| RUN uv sync --locked --no-default-groups --group prod --compile-bytecode | ||
|
|
||
| EXPOSE $PORT | ||
| CMD exec .venv/bin/gunicorn src.server:app \ | ||
| --workers 1 \ | ||
| --worker-class uvicorn.workers.UvicornWorker \ | ||
| --bind "${IP_ADDRESS:-0.0.0.0}:${PORT:-8000}" \ | ||
| --log-level info |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| ## GitHub Projects Discord Bot | ||
|
|
||
| This repository provides an integration between [Discord](https://discord.com/) and [GitHub Projects](https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects), | ||
| allowing users to discuss GitHub Projects directly from Discord. Whenever a new issue, pull request or draft issue is added | ||
| to the project, bot creates a new thread in the specified Discord channel. Every update to the project item is then communicated | ||
| in the thread, allowing users to stay up-to-date with the latest changes. | ||
|
|
||
| ## 🚜 Development | ||
|
|
||
| This repository contains two main components: | ||
| - [`server.py`](/src/server.py): listens for GitHub webhook events and processes them | ||
| - [`bot.py`](/src/bot.py): a Discord bot that creates threads and posts updates | ||
|
|
||
| For local development, you can copy the `.env.example` file to `.env` and fill in the first three environment variables. | ||
| Then run the following command to install dependencies: | ||
|
|
||
| ```bash | ||
| uv sync | ||
| ``` | ||
|
|
||
| To run the server and bot locally, you can use the following command: | ||
|
|
||
| ```bash | ||
| uv run start-app | ||
| ``` | ||
|
|
||
| To run all tests, including unit tests, integration tests and e2e tests, use the following command: | ||
|
|
||
| ```bash | ||
| uv run pytest | ||
| ``` | ||
|
|
||
| ## 🚀 Deployment | ||
|
|
||
| For deployment follow these steps: | ||
| - set all environment variables accordingly, | ||
| - setup file_mount for volumes specified in `docker-compose.yaml` on /app/path_set_in_env, | ||
| - set up a webhook in your GitHub repository to point to your server's `/webhook_endpoint` endpoint, | ||
| - use Dockerfile to build the image. | ||
|
|
||
| ## ⚒️ How it works | ||
|
|
||
| 1. GitHub sends a webhook event to the server when an issue, pull request or draft issue is added or updated in the project. | ||
| 2. The server processes the event and extracts relevant information, such as the issue title, description | ||
| 3. The server updates shared state with the new information which is then used by the bot to post updates. | ||
| 4. The bot creates a new thread in the specified Discord channel for new issues, pull requests or draft issues. | ||
| 5. The bot posts updates in the thread whenever the issue, pull request or draft issue is updated. | ||
| 6. The bot uses the `github_usernames_to_discord_id_mapping.json` file to map GitHub usernames to Discord user IDs, | ||
| allowing it to mention users in the thread. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| services: | ||
| app: | ||
| container_name: app | ||
| build: | ||
| context: . | ||
| dockerfile: Dockerfile | ||
| restart: on-failure | ||
| ports: | ||
| - "${PORT:-8000}:${PORT:-8000}" | ||
| env_file: | ||
| - .env | ||
| volumes: | ||
| - github_id_to_discord_id_mapping:/app/${GITHUB_ID_TO_DISCORD_ID_MAPPING_PATH} | ||
|
|
||
| volumes: | ||
| github_id_to_discord_id_mapping: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| [project] | ||
| name = "github-project-discord-bot" | ||
| version = "1.0.0" | ||
| description = "Create post on forum channel for github project cards" | ||
| requires-python = ">=3.14" | ||
| dependencies = [ | ||
| "aiohttp>=3.13.2", | ||
| "aiorwlock>=1.5.0", | ||
| "dotenv>=0.9.9", | ||
| "fastapi>=0.121.2", | ||
| "hikari>=2.5.0", | ||
| "setuptools>=80.9.0", | ||
| "uvicorn>=0.38.0", | ||
| ] | ||
|
|
||
| [build-system] | ||
| requires = ["setuptools"] | ||
| build-backend = "setuptools.build_meta" | ||
|
|
||
| [tool.setuptools] | ||
| packages = ["src"] | ||
|
|
||
| [dependency-groups] | ||
| prod = [ | ||
| "gunicorn>=23.0.0", | ||
| "uvicorn[standard]>=0.38.0" | ||
| ] | ||
| dev = [ | ||
| "pytest>=9.0.1", | ||
| "pytest-asyncio>=1.3.0", | ||
| "ruff>=0.14.5", | ||
| "pre-commit>=4.4.0", | ||
| "httpx>=0.28.1" # required for testing FastAPI endpoints | ||
| ] | ||
|
|
||
| [tool.uv] | ||
| default-groups = ["dev"] | ||
|
|
||
| [tool.pytest.ini_options] | ||
| testpaths = ["src/tests"] | ||
| asyncio_mode = "auto" | ||
|
|
||
| [project.scripts] | ||
| start-app = "src.main:main" | ||
|
|
||
| [tool.ruff] | ||
| line-length = 120 | ||
| preview = true | ||
|
|
||
| [tool.ruff.lint] | ||
| select = [ | ||
| "E", # pycodestyle errors | ||
| "W", # pycodestyle warnings | ||
| "F", # pyflakes | ||
| "I", # isort | ||
| "B", # flake8-bugbear | ||
| "C4", # flake8-comprehensions | ||
| "UP", # pyupgrade (modern Python syntax) | ||
| "N", # PEP8 Naming | ||
| ] |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| import asyncio | ||
| import os | ||
|
|
||
| from hikari import GuildPublicThread, RESTApp, TokenType | ||
| from hikari.impl import RESTClientImpl | ||
|
|
||
| from src.utils.data_types import ProjectItemEvent | ||
| from src.utils.discord_rest_client import fetch_forum_channel, get_post_id_or_post | ||
| from src.utils.error import ForumChannelNotFound | ||
| from src.utils.github_api import fetch_item_name | ||
| from src.utils.misc import SharedForumChannel, bot_logger, create_item_link, handle_task_exception, retrieve_discord_id | ||
|
|
||
|
|
||
| async def run(state: asyncio.Queue[ProjectItemEvent], stop_after_one_event: bool = False): | ||
| discord_rest = RESTApp() | ||
| await discord_rest.start() | ||
|
|
||
| async with discord_rest.acquire(os.getenv("DISCORD_BOT_TOKEN"), token_type=TokenType.BOT) as client: | ||
| bot_logger.info("Discord client acquired.") | ||
| forum_channel_id = int(os.getenv("FORUM_CHANNEL_ID")) | ||
| discord_guild_id = int(os.getenv("DISCORD_GUILD_ID")) | ||
| forum_channel = await fetch_forum_channel(client, forum_channel_id) | ||
| if forum_channel is None: | ||
| raise ForumChannelNotFound(f"Forum channel with ID {forum_channel_id} not found.") | ||
| shared_forum_channel = SharedForumChannel(forum_channel) | ||
|
|
||
| while True: | ||
| event = await state.get() | ||
| update_task = asyncio.create_task( | ||
| process_update(client, forum_channel_id, discord_guild_id, shared_forum_channel, event) | ||
| ) | ||
| update_task.add_done_callback(lambda task: handle_task_exception(task, "Error processing update:")) | ||
| if stop_after_one_event: | ||
| break | ||
|
|
||
|
|
||
| async def process_update( | ||
| client: RESTClientImpl, | ||
| forum_channel_id: int, | ||
| discord_guild_id: int, | ||
| shared_forum_channel: SharedForumChannel, | ||
| event: ProjectItemEvent, | ||
| ): | ||
| bot_logger.info(f"Processing event for item: {event.node_id}") | ||
|
|
||
| post_id_or_post = await get_post_id_or_post(event.node_id, discord_guild_id, forum_channel_id, client) | ||
| author_discord_id = retrieve_discord_id(event.sender) | ||
| user_mentions = [author_discord_id] if author_discord_id else [] | ||
| user_text_mention = f"<@{author_discord_id}>" if author_discord_id else "nieznany użytkownik" | ||
|
|
||
| if post_id_or_post is None: | ||
| post = await create_post(event, user_text_mention, shared_forum_channel, client, user_mentions) | ||
| elif isinstance(post_id_or_post, int): | ||
| post = await client.fetch_channel(post_id_or_post) | ||
| else: | ||
| post = post_id_or_post | ||
|
|
||
| if not isinstance(post, GuildPublicThread): | ||
| try: | ||
| bot_logger.error(f"Post with ID {post.id} is not a GuildPublicThread.") | ||
| except AttributeError: | ||
| bot_logger.error(f"Post with node_id {event.node_id} is not a GuildPublicThread.") | ||
| return | ||
|
|
||
| message = await event.process(user_text_mention, post, client, shared_forum_channel, forum_channel_id) | ||
| if message: | ||
| await client.create_message(post.id, message, user_mentions=user_mentions) | ||
|
|
||
|
|
||
| async def create_post( | ||
| event: ProjectItemEvent, | ||
| user_text_mention: str, | ||
| shared_forum_channel: SharedForumChannel, | ||
| client: RESTClientImpl, | ||
| user_mentions: list[str], | ||
| ) -> GuildPublicThread: | ||
| bot_logger.info(f"Post not found, creating new post for item: {event.node_id}") | ||
| item_name = await fetch_item_name(event.node_id) | ||
| message = ( | ||
| f"Nowy task stworzony {item_name} przez: {user_text_mention}.\n" | ||
| f" Link do taska: {create_item_link(event.item_id)}" | ||
| ) | ||
| async with shared_forum_channel.lock.reader_lock: | ||
| return await client.create_forum_post( | ||
| shared_forum_channel.forum_channel, | ||
| item_name, | ||
| message, | ||
| auto_archive_duration=10080, | ||
| user_mentions=user_mentions, | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import asyncio | ||
| import os | ||
| from contextlib import asynccontextmanager | ||
|
|
||
| import dotenv | ||
| import uvicorn | ||
| from fastapi import FastAPI | ||
|
|
||
| from src.bot import run | ||
| from src.utils.misc import handle_task_exception | ||
|
|
||
|
|
||
| def main(): | ||
| dotenv.load_dotenv() | ||
| host, port = os.getenv("IP_ADDRESS", "0.0.0.0"), os.getenv("PORT", "8000") | ||
| uvicorn.run("src.server:app", host=host, port=int(port), reload=True) | ||
|
|
||
|
|
||
| @asynccontextmanager | ||
| async def lifespan(app: FastAPI): | ||
| # startup | ||
| app.update_queue = asyncio.Queue() | ||
| bot_task = asyncio.create_task(run(app.update_queue)) | ||
| bot_task.add_done_callback(lambda task: handle_task_exception(task, "Bot task crashed:")) | ||
| yield | ||
| # shutdown | ||
| bot_task.cancel() | ||
| try: | ||
| await bot_task | ||
| except asyncio.CancelledError: | ||
Kubaryt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| pass | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.