Adjuster This! — Learning to correct PV power forecasts with TabPFN and XGBoost, benchmarked against a rule-based OCF adjuster.

This repository provides a modular pipeline to load and audit PV data, engineer features (including TabPFN-TS), train adjusters, and evaluate Mean Absolute Error (MAE) across dates, hours, and horizons. It supports transductive daily retraining (past-week -> current-day) and fixed historical splits.

• Models: TabPFN (time-series & tabular/Reg), XGBoost, AutoGluon (optional), and an OCF-style baseline.
• Evaluation: Per-date, per-hour, per-horizon MAE; overall summaries; export helpers for fixed windows.
• Data Audits: Duplicate & missing (timestamp, horizon) grid checks; optional grid filling.
• Modularity: Clean separation of data loading, preprocessing, feature selection, splits, modeling, and evaluation.
• CLI: run.py to reproduce rolling evaluations with a few flags.

TabAdjust/
├─ core/
│ ├─ feature_selection.py # Permutation importance & selectors (XGB-backed)
│ ├─ preprocessing.py # Model-specific prep; lag/time features
│ └─ splits.py # Holdout builders (rolling & fixed helpers)
│
├─ data/
│ ├─ load.py # CSV loader + duplicate/missing grid audit
│ └─ preprocess.py # High-level preprocessing → ts_data + df
│
├─ evaluation/
│ ├─ export_fixed.py # Helpers to export MAE tables for fixed splits
│ ├─ metrics.py # evaluate_adjusted_forecast / evaluate_ocf_adjuster
│ ├─ runner.py # Main rolling evaluator (daily, TabPFN-TS etc.)
│ └─ final_results/ # Plots & MAE tables produced during studies
│
├─ models/
│ ├─ autogluon_tabular.py # (optional) AG Tabular wrappers
│ ├─ tabadjust.py # Unified dispatcher (TabPFN / XGB / OCF)
│ ├─ tabadjust_ts.py # TS variant glue (TabPFN-TS)
│ ├─ tabpfn_reg.py # TabPFN (Regressor) per-horizon/day runs
│ └─ xgb_reg.py # XGB regressor wrappers & tuned configs
│
├─ utils/
│ └─ utils.py # Pretty tables, diagnostics (optional)
│
├─ outputs/ # Run artifacts (created at runtime)
├─ run.py # CLI entrypoint
├─ .gitattributes # LFS settings (large artifacts)
├─ .gitignore # Ignore AutogluonModels/, checkpoints, etc.
└─ adjuster_dataset_gsoc_v2.csv # Example dataset (schema described below)

Tip: Large training directories (e.g., AutogluonModels/) are ignored by Git; re-train as needed.

Minimum expected columns (UTC timestamps):

• forecast_period_start_datetime_utc (datetime, 30-min cadence)
• forecast_horizon_minutes ∈ {30, 60, …, 480}
• forecasted_pv_generation_MW
• actual_pv_generation_MW
• forecast_error_MW = actual − forecasted (if not present, compute)
• Optional: creation/end timestamps, calendar fields (day-of-week, etc.)

Audits: data/load.py parses timestamps, coerces horizon to Int64, reports exact duplicate (time, horizon) pairs, builds the full half-hour × horizon grid to flag missing keys, and (optionally) appends NaN rows so downstream transforms see a complete key space.

git clone https://github.com/anshulg954/TabAdjust.git
cd TabAdjust

python -m venv adjuster_env
source adjuster_env/bin/activate
pip install -r requirements.txt

It is recommened to use TabPFN-TS or GPU TabPFN(Reg), install a compatible PyTorch build for your platform or using TabPFN-Client from Priorlabs can also serve as an alternative.

Runs the transductive setup (past-week → current-day) across a date range, producing per-date MAE tables.

TabPFN-TS (past-week → current-day)

python run.py \
  --input_csv example.csv \
  --test_start_date 2024-08-01 \
  --test_end_date   2025-05-31 \
  --model_type tabpfn \
  --train_days 7 \
  --use_tabpfn_ts

XGBoost with TabPFN-TS features (same split)

python run.py \
  --input_csv example.csv \
  --test_start_date 2024-08-01 \
  --test_end_date   2025-05-31 \
  --model_type xgboost \
  --train_days 7 \
  --use_tabpfn_ts

Useful flags (rolling mode)

--start_time HH:MM --end_time HH:MM to limit hours (e.g., 04:30–21:00 daylight only)

--max_lag_days 7 adds 7-day lags on top of TS features (if you want hybrid features)

--output_dir outputs controls where artifacts are written

Results are saved to:

outputs/
└─ results/TIMESTAMP/model_type/test_start_date_YYYYMMDD/
   ├─ avg_errors_all_dates.csv
   ├─ avg_errors_per_date.csv
   ├─ avg_errors_per_horizon.csv
   ├─ avg_errors_per_hour.csv
   └─ avg_errors_per_horizon_hour.csv

Use this mode to use fixed historical split for XGB/AG; TabPFN(Reg) uses a rolling lookback per day inside the test period.

In this mode, omit --use_tabpfn_ts. The script will:

• Preprocess raw features,
• Build a fixed train/test split from your date arguments,
• Run the models you select via --models (or --model_type if --models is not provided).

Example usage for XGB + OCF for Jan–Feb 2025 (train: Aug–Dec 2024)

python run.py \
  --input_csv example.csv \
  --train_start_date 2024-08-01 \
  --train_end_date   2024-12-31 \
  --test_start_date  2025-01-01 \
  --test_end_date    2025-02-28 \
  --models xgboost ocf \
  --train_days 15 \
  --max_lag_days 7

Using Add TabPFN(Reg) (rolling 15-day lookback inside the test range)

python run.py \
  --input_csv example.csv \
  --train_start_date 2024-08-01 \
  --train_end_date   2024-12-31 \
  --test_start_date  2025-01-01 \
  --test_end_date    2025-02-28 \
  --models xgboost tabpfn ocf \
  --train_days 15 \
  --tabpfn_min_train_rows 100 \
  --tabpfn_max_train_rows 10000

Using AutoGluon (optional)

python run.py \
  --input_csv example.csv \
  --train_start_date 2024-08-01 \
  --train_end_date   2024-12-31 \
  --test_start_date  2025-01-01 \
  --test_end_date    2025-02-28 \
  --models autogluon ocf \
  --ag_use_gpu \
  --ag_time_limit 1800 \
  --ag_train_path outputs/ag_model \
  --ag_load_path  outputs/ag_model

Daylight-only variant can be used by setting the start-time and the end time

python run.py \
  ... \
  --start_time 04:30 \
  --end_time   21:00

Results are saved to:

outputs/
└─ results/TIMESTAMP>/train_daysd/
   ├─ mae____by_date.csv
   ├─ mae____by_hour.csv
   ├─ mae____by_horizon.csv
   ├─ mae____overall.csv
   └─ rows___.csv        # per-row dump for custom plots

Where is a hyphen-joined list (e.g., xgboost-ocf, xgboost-tabpfn-ocf).

As a part of this project, I ran 12 experiments varying (i) feature space, (ii) train/test policy for XGB/AG, (iii) TabPFN(Reg) lookback, and (iv) daylight filtering (04:30–21:00) using the similar functions in the pipeline.

• Experiments 1–2 (Transductive, daily retrain):
  Both TabPFN-TS and XGB retrain each day on the past week, tested on the current day.
  Features: TabPFN-TS only (Exp. 1), then TabPFN-TS + classical TS + 7-day lags (Exp. 2).

• Experiments 3–12 (Fixed splits for XGB/AG):
  XGB/AG train on a fixed historical split (72/28 → 90/10 in later experiments), tested on a future window;
  TabPFN(Reg) always uses a rolling 15-day lookback and predicts the current day within the same evaluation period.
  Even-numbered experiments apply time filtering (04:30–21:00).
  Features: basic TS + 7-day lags (no TabPFN-TS).

Most important findings (from analysis):

• OCF consistently improves on Baseline across all windows — a robust default.
• TabPFN(Reg) with 15–30 day lookbacks is the most reliable winner in Jan–Apr windows, often 5–9% better than OCF by MAE, especially around midday peaks and across most horizons.
• XGB (Tuned) becomes competitive/winning in May windows (Exp. 11–12), with ~5–6% gains over OCF; earlier windows show higher variance and several OCF wins.
• AutoGluon (default presets) trails in this dataset/split; targeted presets or feature constraints may be needed.

TabPFN (blue) delivers positive improvements in many experiments; XGB (orange) improves in late windows but underperforms OCF in several early ones.

Across a broad set of splits and seasons, OCF reliably improves over Baseline. TabPFN(Reg) with 15–30 day lookbacks is the most dependable, delivering meaningful daytime and horizon-wise MAE reductions. XGB (Tuned) can perform well in the latest windows, however requires large amount of historical data. The pipeline here provides a modular framework to compare, diagnose, and deploy the best adjuster for any PV Forecasting Adjust.

• PVNet (Open Climate Fix)
  Primary repository and documentation for OCF’s multimodal PV forecasting model (NWP + satellite + metadata).
  • PVNet main repo: openclimatefix/PVNet.

• TabPFN-TS (Time-Series wrapper around TabPFN)
  Time-series package that frames forecasting as tabular regression with lightweight feature engineering.
  • Official code: PriorLabs/tabpfn-ts

• TabPFN (Tabular Prior-Data Fitted Networks)
  Foundation model for small/medium tabular tasks; in-context learning via a pretrained Transformer.
  • Original paper: TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (arXiv:2207.01848).
  • Nature paper: Accurate predictions on small data with a tabular foundation model (Nature).
  • Official code: Priorlabs/tabpfn.

Happy Adjusting!