Smart/Agentic Crawler (Invite Collaboration)

[unclecode]

I'm planning to add a smart crawler that takes a set of user-defined objectives and continues crawling to satisfy them. Objectives can be a query requiring a sufficient amount of information to answer, a set of actions with specific goals, or other custom tasks. I also plan to incorporate reinforcement learning into this process later. I've created an issue for discussion and invite interested individuals to share ideas and comments. If you're enthusiastic about contributing, please share your email address, and I'll invite you to our Discord channels to collaborate.

Happy Crawling

[zhixiongtan]

zhixiongtan94@gmail.com
THANKS FOR YOUR WORK

[unclecode]

@zhixiongtan sent :)

[wakaka6]

wakaka6@proton.me
Nice work :) @unclecode

[unclecode]

Sending the links @wakaka6

[wakaka6]

Sending the links   发送链接@wakaka6

@unclecode Hi bro, I didn't get the invite link in my email. Also for the sake of other people contributing code better, it is recommended to create a way for git-hook and github action to use ruff and mypy to check and format the code as a way to ensure the style of the code and avoid confusion caused by editor formatting plugins used by different people.

[unclecode]

@wakaka6 Already sent. and Thank you so much for your suggestion. It has been in our plans for a while. I wanted to engage with everything, like the raw material, to familiarize myself with how people interact with the library and gather ideas for building smart collaboration structures. I completely agree that Ruff is blazing fast, and I believe MyPy is the de facto standard. I love the pre-commit feature because it allows users to check their work on their own machines before submission. We're definitely working on it, and I hope to have it ready by January. May be you join and help me on this too.

[berkaygkv]

This is what I was just thinking. I would be glad to contribute. @unclecode
berkaygokova@gmail.com

[unclecode]

You are most welcome @berkaygkv

[aravindkarnam]

@berkaygkv Sent you an invite link! Check your email.