Why a 1990s Machine Learning Algorithm Destroys LLMs at Predicting House Prices

by Gabriel Quaresma on April 2, 2026 13 min
Table of Contents

Last Updated on July 9, 2026

I asked an LLM how much my neighbor’s house might be worth, a 300m² place with 3 bedrooms, not too old, built in the early 2000s. It gave me a vague, hedged answer that could have applied to almost any property. So I decided to build something better, and in the process, learned an important lesson about when to reach for traditional machine learning instead of a language model.

With all the hype surrounding LLMs, it’s tempting to treat them as silver bullets. But as we’ll see in this post, backed by actual benchmark experiments on real housing data, there are tasks where traditional ML algorithms are not just viable alternatives, but measurably superior. And the best part? The two approaches can work together.

The Problem: How Much Is That House Worth?

Determining a fair price for a property is far from trivial. You can look at when it was built, how big the lot is, its assessed values, and its location, but turning those features into a reliable price estimate requires more than intuition.

An LLM might give you a ballpark figure, but its answer isn’t grounded in your actual dataset. It’s drawing on general training data, and for a task that demands numerical precision on structured inputs, that’s a fundamental limitation.

A more reliable approach is to use a machine learning algorithm designed for exactly this kind of predictive task. In this article, we’ll use Random Forest, train it on real Las Vegas home sales, and then put it head-to-head against an LLM to prove the point with data.

How Random Forest Works

Random Forest combines the output of multiple decision trees to produce a single prediction. Each tree learns from a random subset of the data and makes its own estimate. The forest aggregates all the trees’ answers:

  • For regression (predicting a price), it averages the predictions.
  • For classification (e.g., "cheap" vs. "expensive"), it takes the majority vote.

A single decision tree works by asking a sequence of structured questions about the input data. Starting at the root node, it evaluates a condition (e.g., "Was the house built after 2000?"), branches based on the answer, and repeats until it reaches a leaf node containing the final prediction.

The power of Random Forest lies in combining many such trees. Each one sees a slightly different slice of the data, so their individual errors tend to cancel out. The result is a model that’s more accurate and robust than any single tree.

Defining Our Model

To estimate a property’s price, we need to define features (X) and a target (Y). Instead of a toy dataset, this version uses real residential sales from the City of Las Vegas Open Data Portal (Parcel and Property Assessment Records).

Features:

  • Year built (CONSTYR)
  • Lot size in square feet (LOTSQFT)
  • Lot size in acres (CALC_ACRES)
  • Assessed land value (LANDVAL1)
  • Assessed improvement value (IMPVAL)
  • ZIP code (ZIPCODE)
  • Sale year (derived from SALEDATE)
  • Sale month (derived from SALEDATE)

Target: Sale price (in thousands)

The raw CSV holds 25,000 rows. After keeping only residential sales (SALETYPE == 'R') and dropping rows with missing or invalid core fields, we’re left with 23,075 valid samples to train and evaluate on.

Training the Model in Ruby

We’ll use Rumale (a machine learning library for Ruby) and Numo::NArray (for efficient numerical arrays, similar to NumPy in Python).

require 'csv'
require 'rumale'
require 'numo/narray'

# Load real Las Vegas residential sales
rows = CSV.read("housing_las_vegas_06_22_26.csv", headers: true)

data = rows.filter_map do |row|
  next unless row['SALETYPE'] == 'R'          # residential sales only

  saledate = row['SALEDATE'].to_s             # format: YYYYMMDD
  next if saledate.length != 8
  sale_year  = saledate[0..3].to_i
  sale_month = saledate[4..5].to_i

  features = [
    row['CONSTYR'].to_f, row['LOTSQFT'].to_f, row['CALC_ACRES'].to_f,
    row['LANDVAL1'].to_f, row['IMPVAL'].to_f, row['ZIPCODE'].to_f,
    sale_year.to_f, sale_month.to_f
  ]
  price = row['SALEPRICE'].to_f / 1000.0      # price in thousands

  next if features[0..1].any?(&:zero?) || price.zero?

  features + [price]
end

x = Numo::DFloat.asarray(data.map { |r| r[0..-2] })
y = Numo::DFloat.asarray(data.map { |r| r.last })

# Train Random Forest
model = Rumale::Ensemble::RandomForestRegressor.new(
  n_estimators: 100,  # 100 decision trees
  max_depth: nil,     # let trees grow fully
  random_seed: 42     # reproducible results
)
model.fit(x, y)

# Save for later use
File.open("house_model.dat", "wb") { |f| Marshal.dump(model, f) }

This trains 100 decision trees on the real sales data and saves the model to disk. The random_seed ensures reproducibility: run it twice, and you get the same model.

Once trained, making predictions is straightforward:

require 'rumale'
require 'numo/narray'

def predict_price(params)
  model = Marshal.load(File.read("house_model.dat"))

  input = Numo::DFloat[[
    params[:construction_year],
    params[:lot_sqft],
    params[:calc_acres],
    params[:land_value],
    params[:improvement_value],
    params[:zipcode],
    params[:sale_year],
    params[:sale_month]
  ]]

  puts "Predicted price: #{model.predict(input)[0].round}K"
end

predict_price(
  construction_year: 2005, lot_sqft: 1500, calc_acres: 0.03,
  land_value: 28_000, improvement_value: 65_000, zipcode: 89_149,
  sale_year: 2024, sale_month: 6
)

I wrapped this into a simple web app where users fill in the property features and get an instant price estimate:

The source code is on GitHub. Contributions welcome.

Putting It to the Test: Random Forest vs. LLM

Instead of just claiming that Random Forest is better for this task, I ran a series of controlled experiments to measure the difference across four dimensions: accuracy, latency, consistency, and hybrid integration. You can reproduce every result using the benchmark script included with this post.

Experimental Setup

To make these results reproducible, here are the exact conditions:

Dataset: Real Las Vegas residential sales from the City of Las Vegas Open Data Portal. Of 25,000 total rows, 23,075 remain after filtering to residential sales and dropping invalid records. Each sample has 8 numeric features (year built, lot size in sqft, lot size in acres, assessed land value, assessed improvement value, ZIP code, sale year, sale month) and a target sale price in thousands. Split 70/30 into training (16,152 rows) and test (6,923 rows) sets using a fixed shuffle seed of 42.

Random Forest configuration:

  • Library: Rumale (Ruby)
  • n_estimators: 100
  • max_depth: nil (unlimited)
  • random_seed: 42

LLM configuration:

  • Provider: OpenRouter
  • Model: anthropic/claude-opus-4.6 (Claude Opus 4.6)
  • max_tokens: 50
  • temperature: 1.0

Prompt used (verbatim):

You are a Las Vegas house price estimator. Based on these features, predict the
house price in thousands of dollars (K). Reply with ONLY a number followed by K.
Example: 450K

Property Features:
- Year Built: {year_built}
- Lot Size: {lot_sqft} sqft ({acres} acres)
- Assessed Land Value: ${land_value}
- Assessed Improvement Value: ${improvement_value}
- ZIP Code: {zipcode}
- Sale Date: {sale_month}/{sale_year}

Predicted price:

Evaluation metrics:

  • MAE (Mean Absolute Error): average of |actual − predicted| across test cases
  • RMSE (Root Mean Squared Error): square root of the average squared errors
  • Latency: wall-clock time per prediction (including network round-trip for LLM)
  • Variance: standard deviation across 120 repeated predictions of the same input

The LLM experiments were run on a 120-case test subset to keep API costs manageable. Random Forest metrics are also reported on this same subset for a fair comparison.

Experiment 1: Accuracy

Using the 120-case test subset described above, I ran both models on the same inputs and compared their errors:

MetricRandom ForestLLM (anthropic/claude-opus-4.6)
MAE34.88K ($34,880)220.66K ($220,660)
RMSE55.02K ($55,020)252.46K ($252,460)

The Random Forest model, trained on the actual Las Vegas sales data, produces predictions grounded in the patterns it learned. Its average error (~$35K) is 6.3x lower than the LLM’s (~$221K). The LLM, despite being remarkably capable at language tasks, is essentially guessing based on general knowledge. It has no access to our specific data distribution.

Experiment 2: Latency and Cost

Speed matters, especially if you’re serving predictions in a web app.

MetricRandom ForestLLM
Avg latency0.529 ms2,419.7 ms
Throughput~1,890 predictions/sec~0.41 predictions/sec
Cost per prediction~$0 (local)~$0.05 (API call)
Speedup–~4,576x slower

Random Forest inference is nearly instantaneous: sub-millisecond on a single CPU, and free once the model is trained. An LLM API call involves network round-trips and GPU inference time, making it roughly 4,576x slower and incurring a per-call cost on every prediction.

Experiment 3: Consistency

I asked each model to predict the same house price 120 times.

Random Forest returned the exact same number every time. It’s deterministic given the same input and seed.

The LLM returned a different number on many runs. Even with the same prompt, the stochastic nature of language generation means you get variance in numerical outputs.

Case (year built, lot sqft, land value)RF VarianceLLM Std Dev
1999, 2,419 sqft, $29,7500 (deterministic)5.36K
1996, 1,286 sqft, $28,3500 (deterministic)32.83K
2020, 1,336 sqft, $26,2500 (deterministic)1.88K

For a pricing tool, this inconsistency is a serious problem. Users expect the same input to produce the same output. A house shouldn’t be worth $28K more just because you asked twice.

Experiment 4: The Hybrid Approach

Here’s the twist. Real users don’t type structured assessment records into forms. They describe properties in plain language:

"House built in 2005, 1500 sq ft lot, land value $28,000, improvement value $65,000, in ZIP code 89149"

Random Forest can’t parse that. But an LLM can.

So the production app offers a second mode: instead of extracting features and then separately predicting, the Random Forest model is exposed to the LLM as a callable tool. The LLM reads the description, pulls out the structured parameters, and calls the tool, which runs the actual Random Forest prediction and hands the grounded number back.

User text → LLM (extracts params & calls tool) → Random Forest → Price prediction

In Ruby, using ruby_llm, the model is registered as a tool and the LLM is required to call it:

# The Random Forest model, exposed to the LLM as a callable tool
class HousePredictorLLM < RubyLLM::Tool
  description "Predicts a Las Vegas house price from structured property features"

  # params: construction_year, lot_sqft, calc_acres,
  #         land_value, improvement_value, zipcode

  def execute(construction_year:, lot_sqft:, calc_acres:,
              land_value:, improvement_value:, zipcode:)
    price = HousePredictor.instance.predict(
      construction_year:, lot_sqft:, calc_acres:,
      land_value:, improvement_value:, zipcode:
    )
    { construction_year:, lot_sqft:, calc_acres:,
      land_value:, improvement_value:, zipcode:, predicted_price: price }
  end
end

chat = RubyLLM.chat(model: "anthropic/claude-opus-4.6")
              .with_tool(HousePredictorLLM)

response = chat.ask(<<~TEXT)
  House built in 2005, 1500 sq ft lot, land value $28,000,
  improvement value $65,000, in ZIP code 89149.
TEXT
# The LLM extracts the parameters, calls the Random Forest tool,
# and returns a prediction grounded in the trained model.

Each tool does what it’s best at: the LLM handles flexible human language, the ML model handles numerical prediction. The catch is cost and latency: this convenience runs through the same ~2.4-second, ~$0.05 API call measured in Experiment 2, so it belongs in the user-facing layer, not the high-throughput backend.

What This Teaches Us

The experiments above measured four specific dimensions, and the results point to a clear division of labor:

Where Random Forest wins: On structured numerical prediction, it was more accurate (MAE ~$35K vs. ~$221K, since it actually learned from the data distribution), faster (sub-millisecond vs. ~2.4 seconds per prediction), deterministic (zero variance across repeated calls), and essentially free at inference time. These are not minor advantages. For a production pricing tool, consistency and speed are table stakes.

Where the LLM wins: In Experiment 4, the LLM correctly parsed natural-language descriptions into the structured features the model expects. This is a task that Random Forest simply cannot do. It requires no training data, just the ability to understand human language. Crucially, the LLM’s strength here is understanding the input, not making the prediction. It’s parsing, not estimating.

The broader principle: This isn’t a story about Random Forest being "better" than LLMs. It’s about choosing the right tool for each subtask in a pipeline. LLMs excel at understanding unstructured input and generating structured output. Traditional ML excels at learning patterns from domain-specific data and producing reliable numerical predictions. The hybrid pipeline (LLM as parser, ML model as predictor) leverages both strengths without exposing either to tasks they’re poorly suited for.

This pattern generalizes beyond house prices. Any system where users provide natural-language input but the core task is numerical prediction on structured features (credit scoring, demand forecasting, medical risk assessment, insurance pricing) is a candidate for the same architecture. Before defaulting to an LLM for the entire pipeline, ask: is the hard part understanding the input, or making the prediction? Often it’s the former, and a focused ML model handles the latter better.

Limitations and Caveats

These experiments have known limitations worth acknowledging. The data is real, but it comes from a single market (Las Vegas) over a specific window, so the trained model won’t generalize to other cities without retraining. Two of the strongest features (assessed land value and assessed improvement value) are themselves valuations that track market prices closely, which makes the prediction task easier than starting from raw physical attributes alone; a model given only lot size, age, and location would have a harder time. The LLM had no access to the training data distribution, which is the core reason it underperforms on prediction; in a scenario where an LLM is fine-tuned on the same data or given retrieval access to comparable sales, the accuracy gap would narrow. The latency comparison is also network-dependent: a self-hosted LLM would be faster than an API call, though still orders of magnitude slower than a local Random Forest inference. Finally, we tested a single LLM (Claude Opus 4.6); other models may behave differently on numerical estimation tasks.

What’s Next

A trained model is only useful if it stays current. As new property sales come in, the model should retrain periodically to capture market shifts. In a Rails application, this is straightforward: schedule an ActiveJob that pulls fresh CSV data, retrains the Random Forest, and writes a new house_model.dat, all in the background without interrupting the web app.

Beyond retraining, there are a few natural extensions worth exploring: enriching location beyond ZIP code (neighborhood encoding or latitude/longitude), adding physical attributes like living area, bedrooms, and bathrooms, running feature-importance analysis to understand which variables drive price the most, and caching LLM extractions so repeated natural-language queries don’t each incur a fresh API call.

References

We want to work with you. Check out our Services page!