fix(wikipedia): handle HTTP errors in document comparison and log err… #91
Conversation
There was a problem hiding this comment.
Pull request overview
This PR enhances error handling in the Wikipedia updater workflow by specifically catching and logging HTTP errors that occur during document comparison operations.
Changes:
- Added HTTP error handling to catch
requests.exceptions.HTTPErrorduring document comparison - Implemented database logging for HTTP errors via
ErrorRetrievalentries
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| db_session.add( | ||
| ErrorRetrieval( | ||
| document_id=wld.id, | ||
| http_error_code=e.response.status_code, |
There was a problem hiding this comment.
Potential AttributeError if e.response is None. The HTTPError exception doesn't guarantee that the response attribute is set. Add a guard to handle cases where e.response might be None, such as network timeouts or connection errors.
| db_session.add( | |
| ErrorRetrieval( | |
| document_id=wld.id, | |
| http_error_code=e.response.status_code, | |
| http_error_code = e.response.status_code if e.response is not None else None | |
| db_session.add( | |
| ErrorRetrieval( | |
| document_id=wld.id, | |
| http_error_code=http_error_code, |
| db_session.add( | ||
| ErrorRetrieval( | ||
| document_id=wld.id, | ||
| http_error_code=e.response.status_code, | ||
| error_info=f"HTTP error occurred: {str(e)} from wikipedia_updater", |
There was a problem hiding this comment.
The error message format 'HTTP error occurred: {str(e)} from wikipedia_updater' could be more informative. Consider including the URL or endpoint that failed and the specific HTTP status code in a structured format for better debugging.
| db_session.add( | |
| ErrorRetrieval( | |
| document_id=wld.id, | |
| http_error_code=e.response.status_code, | |
| error_info=f"HTTP error occurred: {str(e)} from wikipedia_updater", | |
| response = getattr(e, "response", None) | |
| status_code = getattr(response, "status_code", "unknown") | |
| request_url = getattr(response, "url", "unknown") | |
| db_session.add( | |
| ErrorRetrieval( | |
| document_id=wld.id, | |
| http_error_code=status_code, | |
| error_info=( | |
| f"HTTPError in wikipedia_updater | " | |
| f"status_code={status_code} | " | |
| f"url={request_url} | " | |
| f"detail={e}" | |
| ), |
This pull request introduces improved error handling for HTTP errors in the
WikipediaUpdaterworkflow. Now, when an HTTP error occurs during document comparison, the error is logged and a corresponding entry is added to the database for better traceability.Error handling improvements:
requests.exceptionsto support handling HTTP errors in the workflow (wikipedia_updater.py).main()function to catchrequests.exceptions.HTTPError, log the error, and create anErrorRetrievalentry in the database with details about the HTTP error (wikipedia_updater.py).