Immowelt Scraper

A Python scraper for Immowelt.de property listings using the ScrapingAnt API. It extracts detailed property information including price, location, size, energy rating, and more.

Features

• Scrape property listings from any supported German city
• Support for multiple property types (houses, apartments, land, commercial, etc.)
• Parallel execution for fast scraping
• Extract 13 property attributes per listing
• CSV export with German number formatting handled
• Residential German proxies for reliable scraping

Requirements

• Python 3.10+
• ScrapingAnt API key

Installation

1. Clone this repository or download the files

2. Create a virtual environment (recommended):

   python3 -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Add your ScrapingAnt API key to config.py:

   SCRAPINGANT_API_KEY = "<YOUR_SCRAPINGANT_API_KEY>"

Obtaining ScrapingAnt API Token

1. Go to ScrapingAnt
2. Click Sign Up or Get Started Free
3. Create an account using email or Google/GitHub OAuth
4. After registration, navigate to your Dashboard
5. Find your API key in the API Key section
6. Copy the API key and paste it into config.py

Note: ScrapingAnt offers a free tier with limited API credits. For production usage, consider upgrading to a paid plan.

Usage

Basic usage (houses in Hamburg):

python main.py hamburg

Scrape apartments in Berlin:

python main.py berlin --type apartment

Limit number of properties:

python main.py munich --limit 50

Custom output file:

python main.py cologne -o cologne_properties.csv

Verbose mode with custom workers:

python main.py frankfurt --verbose --max-workers 5

All options:

python main.py --help

Command Line Arguments

Argument	Description	Default
city	City name to search	Required
--type, -t	Property type	house
--output, -o	Output CSV file	{city}{type}{date}.csv
--limit, -l	Max properties to scrape	None (all)
--max-workers, -w	Parallel requests	5
--verbose, -v	Enable debug output	False

Supported Cities

Berlin, Hamburg, Munich, Cologne, Frankfurt, Stuttgart, Dusseldorf, Dortmund, Essen, Leipzig, Bremen, Dresden, Hanover, Nuremberg, Duisburg

Supported Property Types

Type	Description
house	Houses (Häuser)
apartment	Apartments (Wohnungen)
land	Land plots (Grundstücke)
commercial	Commercial properties
office	Office spaces
gastronomy	Gastronomy businesses

Extracted Data

Each property listing includes:

Field	Description
url	Property listing URL
price	Purchase price in EUR
price_per_sqm	Price per square meter
living_area	Living area in m²
plot_size	Plot size in m²
location	District and city
postal_code	German postal code
property_type	Type of property
year_built	Year of construction
energy_rating	Energy efficiency class (A+ to H)
heating_type	Type of heating system
energy_source	Energy source
commission	Buyer's commission

Output

The script will:

1. Fetch the first search page to get total listings count
2. Fetch remaining pages in parallel to collect all property URLs
3. Scrape each property detail page in parallel
4. Export results to CSV file
5. Print a summary with property count

Example output:

==================================================
Scraping Complete!
==================================================
City: Hamburg
Property Type: house
Properties Found: 1641
Output File: hamburg_house_20251220.csv
==================================================

How It Works

1. Builds search URL with city code and property type filters
2. Fetches first page via ScrapingAnt API with German residential proxies
3. Extracts total count and property URLs from search results
4. Fetches remaining pages in parallel using ThreadPoolExecutor
5. Collects all unique property expose URLs
6. Scrapes each property detail page in parallel
7. Parses HTML to extract all property attributes
8. Exports structured data to CSV

Troubleshooting

• 423 errors: The scraper uses residential German proxies to avoid bot detection
• Missing data: Some fields may be empty if not provided by the seller
• Rate limiting: Reduce --max-workers if you encounter 429 errors
• API errors: Verify your API key is correct and has available credits

License

MIT License