IdeaWell

Project ideas from Hacker News discussions.

Better HTTP/1 connection reuse in Go 1.27

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Theme 1 – New automatic body‑draining behavior and its trade‑offs
The change makes Response.Body.Close() read (up to ~50 ms) the remaining bytes so connections can be reused safely, but it also means you can no longer skip downloading the body by closing early.
- “It sucks that this changes the behaviour under the hood … but I do like the tradeoff here, since it also limits the download time to just 50 ms … you'll just wait a little bit of traffic instead.” – nasretdinov
- “I can attest to the fact this is a Frequently Asked Question among people who are trying to Go as an HTTP client… this feels somewhat icky, I can understand where it is coming from.” – jerf

Theme 2 – Need for clear documentation
Many participants noted that the semantics of Close() are not obvious and should be documented in the net/http package itself.
- “Am I missing something or is none of this information documented in net/http itself? This is a surprising implementation of 'Close' and should be documented somewhere.” – jerf
- “Note, however, that Transport will automatically try to read a Response Body to EOF asynchronously up to a conservative limit when a Body is closed.” – aleksi (link to https://pkg.go.dev/net/http#Client.Do)
- “I think it is reasonable for it to not be documented directly on Close … if you really wanted to slam the connection closed ASAP it does actually seem more logical to grab the underlying connection …” – jchw

Theme 3 – Interaction with HTTP/2, proxies, and internal service traffic
The behavior matters most for HTTP/1 (especially internal microservices); HTTP/2 already avoids the need for draining, and reverse‑proxy setups can see side‑effects if bodies aren’t consumed.
- “As explained in the post, you don’t need any of this in HTTP/2, since you get the benefits of connection sharing without having to read the drain connection body by protocol design.” – giovannibajo1
- “At the very end it mentions it won't drain bodies for Connection: close … Does this work with reverse proxies? I've had issues with nginx still getting IO errors …” – prussian
- “Although the article demonstrates the changes by making requests of a public web server, I would imagine this change has been primarily motivated by a desire to reduce overhead and latency on internal communication between microservices…” – MrDOS

🚀 Project Ideas

Generating project ideas…

[httpclose – Early‑Connection‑Close Helper for Go]

Summary

  • A tiny Go library that lets you close an HTTP connection without forcing the Transport to drain the response body.
  • Provides CloseWithoutDrain(resp *http.Response) error which hijacks the underlying net.Conn and closes it instantly, avoiding the automatic async read‑up‑to‑limit behavior.
  • Core value: eliminates latency spikes and unwanted bandwidth consumption when you only need to abort a request (e.g., canceling a long‑poll or microservice health check).

Details

Key Value
Target Audience Go developers building HTTP clients, especially those dealing with microservices, reverse‑proxies, or latency‑sensitive loops
Core Feature CloseWithoutDrain function that safely obtains and closes the raw connection from a *http.Response
Tech Stack Go stdlib (net/http, net), optional use of debug/buildvcs for versioning; no external deps
Difficulty Low
Monetization Hobby

Notes

  • HN commenters complained about the hidden async drain (jerf: “if I want to slam this connection closed … how do I do that without uselessly draining a socket?”) – this library directly answers that need.
  • Enables discussion around proper connection lifecycle control and can be paired with existing tracing tools to measure saved latency.

[httpdrain‑observer – Middleware to Log Automatic Body Drains]

Summary

  • A transport‑level middleware that wraps http.RoundTripper to detect when Go’s Transport automatically drains a response body (the async read up to the conservative limit) and emits structured logs/metrics.
  • Helps teams quantify the hidden cost of resp.Body.Close() in production, especially for high‑QPS internal services.
  • Core value: visibility into an otherwise opaque behavior, allowing performance tuning or informed decision‑making about disabling HTTP/2, tweaking limits, or adopting explicit close patterns.

Details

Key Value
Target Audience Backend engineers, SREs, and performance‑focused teams using Go HTTP clients at scale
Core Feature Wrapper Transport that increments a counter and records latency whenever the Transport performs an automatic drain; integrates with Zap, Zerolog, or Prometheus
Tech Stack Go stdlib, optional go.uber.org/zap, prometheus/client_golang
Difficulty Medium
Monetization Hobby

Notes

  • Directly addresses jerf’s request for documentation and visibility: “Am I missing something or is none of this information documented in net/http itself?” – this tool surfaces the behavior.
  • Can spark HN discussions about trade‑offs between automatic draining and manual control, and provide concrete data for teams deciding whether to disable HTTP/2 or adjust limits.

[genhttpclose – Code‑Gen Utility for Boiler‑plate Close‑Without‑Drain Helpers]

Summary

  • A command‑line tool that scans Go source files for patterns like resp.Body.Close() and generates a tailored helper function (or method) that closes the underlying connection without triggering the automatic drain, using reflection‑safe techniques.
  • Outputs ready‑to‑copy code snippets or a generated .go file that projects can import, eliminating copy‑paste fatigue and bug‑prone manual updates.
  • Core value: reduces repetitive boilerplate and the risk of forgetting to update helpers when the stdlib changes, directly addressing the pain of “copying code from project to project”.

Details

Key Value
Target Audience Go developers who frequently write HTTP client code and maintain internal libraries or microservices
Core Feature AST‑based scanning (go/ast, go/parser) to find Body.Close() calls and emit a CloseWithoutDrain(resp *http.Response) error helper
Tech Stack Go stdlib (go/ast, go/parser, go/token), Cobra for CLI
Difficulty Medium
Monetization Hobby

Notes

  • HN users (jamesponddotco, aleksi) expressed frustration about copying a helper across projects and the maintenance burden when a bug is found: “It was a pain to update everything when I inevitably found a bug in what I copied.” This tool automates that helper generation.
  • Encourages community discussion on best practices for connection management and could be extended to support other HTTP client patterns (e.g., custom timeouts, context cancellation).

Read Later