Nicheloom visits: 60,536 all-time · 7,913 in the last 7 days

The opportunity tracker for new startups.

Today: 895 launches · Niche of the day: browser-based video games · See what launched today →

The Hessian of tall-skinny networks is easy to invert

Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.

Want to reach these readers? Sponsor an issue.

It turns out the inverse of the Hessian of a deep net is easy to apply to a vector. Doing this naively takes cubically many operations in the number of layers (so impractical), but it's possible to do this in time linear in the number of layers (so very practical)!This is possible because the Hessian of a deep net has a matrix polynomial structure that factorizes nicely. The Hessian-inverse-product algorithm that takes advantage of this is similar to running backprop on a dual version of the deep net. It echoes an old idea of Pearlmutter's for computing Hessian-vector products.Maybe this idea is useful as a preconditioner for stochastic gradient descent?

HN · launched Jan 15, 2026 · 31 upvotes (top 28%) · 0 views

Niche: mathematical solvers and optimization engines · 80 launches, 2 in the last 7 days · Explore this niche →

Category and tags

  • Horizontal
  • Model & infra
  • Developer
  • AI-native
  • Hobby / open-source project

Visit website ↗ Source listing ↗

Could you build this?

No This is a mathematical machine learning research project deriving and proving linear-time Hessian inversion algorithms for deep neural networks, representing novel academic computer science research.

What it would actually take: Implementing this requires deriving exact second-order gradient formulations for specific neural network topologies and implementing custom matrix factorizations (e.g. specialized Woodbury matrix identity variants or backpropagation through Kronecker-factored approximations). It requires a research scientist with a PhD-level background in theoretical deep learning, optimization theory, and high-performance linear algebra (C++/CUDA).

Discussion

20 comments analyzed.

Competitors mentioned: BFGS, Conjugate Gradient (CG), SGD-type optimizers, Levenberg-Marquardt, Jacobian-free Newton-Krylov methods

Concerns raised: Situation dependent - no free lunch theorem applies, Hessian often not invertible without regularization, Computational and memory overhead vs. gradient descent, Requires model architecture and tricks to be worthwhile on larger models

Feature requests: Training runs and experimental results, Comparison with Conjugate Gradient methods, Better educational resources explaining Hessians and Jacobians

Competitors

Other products that read as similar to this one : 285 launches clear the similarity bar, closest 8 shown.

Attention rank: #76 of 286 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).

Launched 72 days after the earliest competitor.

The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.