Handle empty search

[DinhLongHuynh]

Empty DataFrame Handling Issue in ArticlesProcessor

Problem Overview

When no papers are found on a specific day, the search returns an empty list, which gets converted to an empty DataFrame in the ArticlesProcessor class. This causes errors in all processing methods because they assume the DataFrame has the expected structure and columns.

Error Flow

1. Empty Search Results → articles = []
2. Empty DataFrame Creation → pd.DataFrame.from_dict([]) creates DataFrame with no columns
3. Processing Methods Fail → All methods assume specific columns exist

Specific Errors

• KeyError: "None of [Index(['databases', 'publication_date', 'title', 'keywords', 'url'], dtype='object')] are in the [columns]"
• AttributeError: 'Series' object has no attribute 'apply'
• IndexError: When accessing non-existent columns

Root Cause

• No validation layer in processing methods
• Assumption of data existence - code assumes papers will always be found

Solution

Add validation layer in each ArticlesProcessor function to:

1. Check if DataFrame is empty before processing
2. Initialize expected structure if empty
3. Use defensive programming to handle edge cases
4. Maintain consistent output format regardless of input

This ensures robust and reliable operation even when no papers are found.

[VladimirShitov]

Hey @DinhLongHuynh , thank you so much for the contribution! I will test it and happily merge asap if there are no problems

[VladimirShitov]

Thank you, looks great! I simplified the code a bit, but it should yield the same result. I've also added a test case with an empty paper list in telegram