IdeaWell

Project ideas from Hacker News discussions.

google.com/goto: Google's anti-scraping update

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

  1. Privacy and tracking concerns: Users worry the change prevents seeing destination URLs before clicking, enabling non-bypassable tracking. As JohnFen states: "Because I wouldn't want Google to know what links I click on." orlp adds: "2. It is non-bypassable tracking of every single click."

  2. User experience degradation: Redirect URLs cause noticeable delays and frustration. The original poster notes: "Sometimes, these redirect URLs take a perceivable amount of time to load, which is very irritating." Maxion confirms: "Can stall up to 20 seconds+ sometimes."

  3. Impact on scraping and archiving: The change breaks tools relying on direct URLs and increases scraping costs. InsideOutSanta explains: "It's primarily relevant because it makes scraping search results much more expensive, solidifying Google's effective monopoly on Internet search." leobg adds: "In effect, they're removing attribution from the content they quote from other people's websites."

  4. Shift to alternative search engines: Many users are migrating to services like Kagi, ChatGPT, or Brave Search. tarkin2 says: "They are forcing my further and further towards kagi." JohnFen shares: "Kagi is what I settled on a couple of years ago. I do think sometimes the fawning is over the top, but it is a solid search engine..."

  5. Google's business motivations: Critics view the change as anti-competitive "enshittification" to protect Google's monopoly. nullbio states: "The enshittification continues. Nothing good ever comes from businesses desperately tryinging to protect their moats rather than making their products better." InsideOutSanta argues: "Google has previously tried to prevent scraping of search results using legal means... This is Google's reaction to that."

🚀 Project Ideas

Generating project ideas…

RedirectResolver Extension

Summary

  • A lightweight browser extension that intercepts Google /goto and /url redirect links, resolves the final destination URL in the background, and rewrites the href attribute so users see and can copy the real link instantly.
  • Eliminates perceptible redirect latency and restores the ability to hover, right‑click, and copy clean URLs without Google tracking.

Details

Key Value
Target Audience Power users, privacy‑conscious searchers, and anyone frustrated by Google’s delayed redirect links
Core Feature On‑the‑fly decoding of Base64‑protobuf redirect URLs and replacement with the target URL before the user interacts with the link
Tech Stack Manifest V3 (Chrome/Firefox), Rust‑wasm for protobuf parsing, optional Service Worker for caching resolutions
Difficulty Medium
Monetization Hobby

Notes

  • HN users complained about “perceivable amount of time to load” and wanting to “see the direct link on the bottom of the screen when you hover” (1e1a, darknavi). This extension directly addresses those frustrations.
  • By providing instant, copy‑able URLs it also restores basic hypertext expectations, a point raised by plusieurs commenters who miss the old behavior.

DirectLink API Service

Summary

  • A simple HTTP API that accepts a Google redirect URL (e.g., www.google.com/goto?url=…) and returns the resolved destination URL, handling the protobuf decoding and optional caching.
  • Enables developers, scrapers, and alternative search frontends to bypass Google’s obfuscation without hitting rate limits or exposing user clicks.

Details

Key Value
Target Audience Developers building search tools, archival services (e.g., Wayback Machine), and privacy‑focused metasearch engines
Core Feature Resolve Google /goto links to plain URLs via a REST/JSON endpoint, with automatic caching of recently seen redirects
Tech Stack Go or Rust backend, protobuf decoding library, Redis cache, deployed on Fly.io or similar low‑cost platform
Difficulty Low
Monetization Revenue-ready: $5/mo for 100k resolutions, free tier for hobby use

Notes

  • Commenters noted that extensions like ClearURLs no longer work because the URL is encrypted (joeblubaugh, Dylan16807). An API offers a programmatic workaround that can be integrated into any tool.
  • Provides a way to “get a link to paste into archive.org” when the redirect fails (tgsovlerkhgsel), directly addressing a concrete use case mentioned in the thread.

CleanSearch Frontend

Summary

  • A minimalist search UI that proxies Google (or other engines) requests, strips all redirect and tracking parameters, and presents results with direct, copy‑able URLs and optional toggles to hide AI summaries.
  • Gives users a Google‑like experience without the latency, tracking, or AI‑clutter that many find annoying.

Details

Key Value
Target Audience Users who want plain web results, dislike Google’s AI overlays, and seek a faster, tracker‑free search page
Core Feature Fetch search results, strip /url//goto redirects, strip udm=… tracking params, and optionally hide AI/generated snippets
Tech Stack Next.js (React) frontend, Vercel edge functions for proxying, optional integration with Bing API or SearXNG as upstream
Difficulty Medium
Monetization Hobby (can be self‑hosted) or Revenue‑ready: donations via OpenCollective for hosted instance

Notes

  • Many commenters expressed a desire for “just a damn search engine that works” and lamented AI results clutter (lukeify, insideoutsanta). CleanSearch delivers exactly that by stripping the unwanted layers.
  • By preserving the original URLs it also satisfies the need to “copy the link and send it to someone” without sharing a Google tracking link (Cider9986).

LinkInspector Bookmarklet

Summary

  • A drag‑to‑bookmark bookmarklet that, when activated on a Google results page, scans all result links, resolves any redirect URLs, and displays the clean destination URL next to each result or in a tooltip.
  • No installation required; works in any browser and respects user privacy by performing resolutions locally.

Details

Key Value
Target Audience Casual users who dislike extensions but want occasional ability to see real URLs without leaving the page
Core Feature One‑click activation to replace or annotate obfuscated Google links with their true URLs on the current page
Tech Stack Pure JavaScript (ES6), optional protobuf parsing library, works as a bookmarklet
Difficulty Low
Monetization Hobby

Notes

  • Users frequently asked for a way to “see the direct link on the bottom of the screen when you hover” and to “copy the link” without visiting the redirect (1e1a, darknavi). The bookmarklet offers an instant, zero‑setup solution.
  • It also sidesteps the issue of extensions being broken by Google’s encryption, since the logic runs on the page itself after the user triggers it.

MetaSearch Aggregator with Direct Links

Summary

  • A metasearch tool that queries multiple search APIs (Google via udm=web, Bing, DuckDuckGo, Yandex, etc.), deduplicates results, and presents a unified list where every entry shows the direct URL and offers a “copy link” action.
  • Optionally filters out AI-generated summaries and lets users weight or block specific domains (a feature praised in Kagi).

Details

Key Value
Target Audience Power searchers, researchers, and privacy‑advocates who want unbiased, comprehensive results without Google’s tracking or AI clutter
Core Feature Aggregate, deduplicate, and display search results with clean URLs; provide domain weighting and AI‑result toggle
Tech Stack Python/FastAPI backend, async HTTP clients to each search API, React frontend, deployed on Render or Docker
Difficulty Medium
Monetization Revenue‑ready: $4/mo for unlimited searches, free tier limited to 50 searches/day

Notes

  • Commenters praised Kagi’s site‑weighting and blocking features but noted the cost (lukeify, insideoutsanta). This aggregator offers similar controls for free or at a lower price.
  • By providing direct URLs it resolves the core pain point of “cannot copy the link from the Google result” (Cider9986) and mitigates tracking concerns highlighted throughout the discussion.

Read Later