Skip to content

Conversation

@smstone0
Copy link
Member

@smstone0 smstone0 commented Jul 24, 2025

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

What

Cleaned up the repository following the Copilot dashboard migration to remove old files, dependencies, and Terraform.

Made several improvements e.g. refactoring lambda, adding tests, fixing linting checks, adding a license, getting linting and testing to run in CI.

Testing

Have any new tests been added as part of this issue? If not, try to explain why test coverage is not needed here.

  • Yes
  • No
    Please write a brief description of why test coverage is not necessary here.
  • Not as part of this ticket. (Could be done at a later point)

Documentation

Has any new documentation been written as part of this issue? We should try to keep documentation up to date
as new code is added, rather than leaving it for the future.

  • Yes
  • No
    Please write a brief description of why documentation is not necessary here.
  • Not as part of this ticket. (Could be done at a later point)

Related issues

KEH-921

How to review

  • View and test new commands in the Makefile.
  • Run lambda.

@smstone0 smstone0 marked this pull request as draft July 24, 2025 13:03
@smstone0 smstone0 changed the title KEH-921 KEH-921 - Post-Migration Cleanup Aug 5, 2025
@smstone0 smstone0 requested a review from a team August 5, 2025 12:53
@smstone0 smstone0 marked this pull request as ready for review August 5, 2025 12:53
Copy link
Contributor

@TotalDwarf03 TotalDwarf03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minors. Good work 👍

README.md Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A link to the action that does this might be useful

README.md Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only runs on src now

@smstone0
Copy link
Member Author

smstone0 commented Aug 6, 2025

All comments above should now be addressed

@smstone0 smstone0 requested a review from TotalDwarf03 August 6, 2025 11:12
Copy link
Contributor

@TotalDwarf03 TotalDwarf03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of comments still haven't been addressed.

  • Docstrings + function type hints in main.py
  • Backticks in README.md for directory names
  • Need to remove sdp-sandbox refs

@sebtheo sebtheo requested a review from a team August 7, 2025 10:37
Copy link
Contributor

@sebtheo sebtheo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RIP dashboard. I haven't run locally but presume you have deployed to sdp-dev to test?

Copy link
Contributor

@TotalDwarf03 TotalDwarf03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. One minor comment to do but I'll approve in advance.

Good work on this 👍

src/main.py Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json isn't a valid type I don't think.

Believe this should a list[dict]

@smstone0 smstone0 merged this pull request into master Aug 7, 2025
4 checks passed
@smstone0 smstone0 deleted the KEH-921 branch August 7, 2025 14:56
TotalDwarf03 added a commit that referenced this pull request Aug 22, 2025
* Remove dashboard Terraform

* Remove dashboard related files

* Update Poetry and remove Streamlit

* Remove pandas, matplotlib, plotly

* Update and move CODEOWNERS to .github

* Copy lambda README to root

* Remove dashboard Dockerfile

* Update README prerequisites and add makefile and table of contents sections

* Remove dashboard dockerignore

* Rename lambda folder to src

* Add Makefile commands for applying linting

* Run linters

* Improve README formatting

* Refactor lambda
- Fix var typo
- Capitalise constants
- CoPilot -> Copilot
- Add missing docstring
- Remove unused var

* Move terraform

* Refactor lambda
- Add module docstring
- Use isinstance check
- Use lazy formatting for logging
- Remove unnecessary else and function parameter

* Create LICENSE

* Fail CI if linting fails

* Potential fix for code scanning alert no. 5: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Remove old info from docs

* Refactor lambda
- Use Optional typing
- Get Pylint to ignore handler parameters
- Extract repeated logic for updating S3
- Extract logic for converting data to dictionary

* Reduce linter max line length to 100 to fix Pylint conflict

* Replace try except blocks with response type checks

* Split handler into separate functions: reached 10/10 on pylint

* Add tests folder

* Update Makefile
- Clean more temp files
- Run linters on tests folder
- Add new command for running tests

* Update README
- Add table of contents
- Add testing and linting info

* feat: update and setup for testing + coverage.

- Added pytest-cov and pytest-xdist for enhanced testing capabilities.
- Updated Makefile to include new test commands.
- Modified .gitignore to exclude coverage reports.
- Update README.md table of contents.

* chore: update makefile commands

- Remove tests directory from linting and formatting commands in Makefile.
- Add .coverage to clean command in Makefile.

* test: Write some tests for main.py

- Test get_team_history
- Test get_and_update_copilot_teams
- Test update_s3_object
- Add VS Code settings to gitignore

* chore: remove old commented out tests

* Add missing tests to reach 100% coverage

* Add tests to ci.yml

* Fix failing tests

* Install dependencies via Poetry from requirements txt files and update all dependencies

* Group Poetry docs dependencies

* Use Poetry for mkdocs
- Delete requirements txt file
- Update deployment workflow
- Update setup documentation

* Fix Docker run command formatting

* Delete requirements.txt

* refactor: Update Dockerfile to use Poetry

- Moved Dockerfile to root directory for better visibility.
- Update Dockerfile to use Poetry for dependency management instead of pip.
- Update .dockerignore to exclude unnecessary files from build context.
- Update README.md to reflect move of Dockerfile to root directory.

* Add option to run lambda outside of a container
- Update README with setup instructions
- Add commented function call to main.py

* Address comments
- Specify where to find CI workflow
- Update sentence saying linting auto runs on tests
- Fix README env vars
- Use poetry install --only docs for mkdocs deployment
- Add missing docstrings and return types
- Return boolean for update_s3_object
- Add comment above dev code

* Run linting

* Update README and .dockerignore
- Remove disclaimer from README and update overview section with dashboard repository link
- Add .github/ to .dockerignore

* Remove and replace sandbox references
- Remove and replace sandbox from .sh files, README, example_tfvars, variables.tf
- Delete sandbox Terraform and move its example_tfvars to dev folder

* Add backticks to README

* Address comments
- Use lambda full name
- Update documentation bash
- Move handler function to bottom of script

* Address comments
- Move logger to end of handler function
- Remove Digital Landscape links
- Specify Docker endpoint should return message if successful

* Add missing type hints and return types

* Tidy lambda scripts info in README

* Fix function return type

---------

Co-authored-by: Kieran Pritchard <kieran.pritchard@ons.gov.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants