Nginx upstream sent too big header while reading response header from upstream

last updated in Categories , , ,

Nginx upstream sent too big header while reading response header from upstream

I am getting the following error and my app stops working:
    2020/07/12 14:32:03 [error] 45625#45625: *10918 upstream sent too big header while reading response header from upstream, client: 139.xxx.yyy.zzz, server: www.nixcraft.com, request: “POST /app/mg/posts HTTP/2.0”, upstream: “fastcgi://unix:/run/php/php-fpm.sock:”, host: “www.nixcraft.com”, referrer: “https://www.nixcraft.com/mg/admin/index.app”
How do I fix this error for Nginx web server running on Linux or Unix-like systems?

This page explains how to fix Nginx error, upstream sent too big header while reading response header from upstream on Linux or Unix system.

ADVERTISEMENTS


Nginx upstream sent too big header while reading response header from upstream

This error caused by using Nginx in reverse proxy mode or FastCGI is configured for PHP/Python/Perl and other apps. You need to enable the buffering of responses from the proxied server. Nginx designed to accelerate requests. Hence, it will buffer all requests made to a backend server such as Apache or FastCGI process such as PHP-FPM. By default, the buffer size is equal to one memory page. For example, 4K or 8K, depending on operating systems. If more information sent nginx will cry out with an error in your log files:
$ tail -f /var/log/nginx/www.nixcraft.com_error.log

upstream sent too big header while reading response header from upstream

Fix when Nginx is running in a proxy / reverse proxy mode

Edit your nginx.conf or virtual domain file:
$ sudo vi /etc/nginx/vhosts.d/nixcraft.com.conf
Set the following in http or server or location contaxt:

server {
 proxy_busy_buffers_size   512k;
 proxy_buffers   4 512k;
 proxy_buffer_size   256k;
 # rest of nginx config #
}

Make sure your test and reload nginx server:
# nginx -t
# nginx -s reload

Where,

  1. proxy_busy_buffers_size : When buffering of responses from the proxied server is enabled, limits the total size of buffers that can be busy sending a response to the client while the response is not yet fully read. In the meantime, the rest of the buffers can be used for reading the response and, if needed, buffering part of the response to a temporary file. By default, size is limited by the size of two buffers set by the proxy_buffer_size and proxy_buffers directives.
  2. proxy_buffers : Sets the number (4) and size (512k) of the buffers used for reading a response from the proxied server, for a single connection.
  3. proxy_buffer_size : Sets the size of the buffer used for reading the first part of the response received from the proxied server.

Can I turn off proxy buffering?

Yes, but I don’t recommend it for performance reasons. However, here are config directives to turn it off:

######################################################################################
## 'proxy_buffering off|on;' must be set in http, or server, or location directives ##
######################################################################################
http {
    proxy_buffering off;
    # rest of config
}

Dealing with proxy_pass and fastcgi buffers

Consider the following PHP-fmp config:

## set php as server path ##
upstream php {
  server unix:/run/php/php-fpm.sock;
}
    # Pass all .php files onto a php-fpm/php-fcgi server.
     index index.php index.html;
     location ~ [^/]\.php(/|$) {
      fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
      fastcgi_pass php;
      fastcgi_split_path_info ^(.+?\.php)(/.*)$;
            if (!-f $document_root$fastcgi_script_name) {
                     return 404;
     }

We need to append the following directives after fastcgi_pass:

    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass php;
    ## TUNE buffers to avoid error ##  
    fastcgi_buffers 16 32k;
    fastcgi_buffer_size 64k;
    fastcgi_busy_buffers_size 64k;

Where,

  1. fastcgi_buffers : Sets the number (16) and size (32k) of the buffers used for reading a response from the FastCGI server, for a single connection
  2. fastcgi_buffer_size : We can sets the size of the buffer used for reading the first part of the response received from the FastCGI server.
  3. fastcgi_busy_buffers_size : When buffering of responses from the FastCGI server enabled, limits the total size of buffers that can be busy sending a response to the client while the response is not yet fully read. In the meantime, the rest of the buffers used for reading the response and, if needed, buffering part of the response to a temporary file.

Can I turn off fastcgi buffering?

Again, yes, but I don’t recommend turning it off for performance reasons. However, here are config directives for your ready reference:

##
## Enables or disables buffering of responses from the FastCGI server ##
## Must be set in http, server, location contaxt 
## fastcgi_buffering on | off;
##
location / {
   fastcgi_buffering off;
   # rest of config below #
}

Can I combine both fastcgi and proxy buffering?

Yes, I add the following in my http contaxt to stay safe:

http {
 fastcgi_buffers 16 32k;
 fastcgi_buffer_size 64k;
 fastcgi_busy_buffers_size 64k;
 proxy_buffer_size   128k;
 proxy_buffers   4 256k;
 proxy_busy_buffers_size   256k;
 # rest of config #
}

Other suggesions

Make sure upstream PHP/Python/Perl/Ruby server is not crashed. Check if the upstream Apache server sent an invalid header back to Nginx. Make sure your server not under DoS/DDoS attack. Those can cause many errors and overflow buffer. Always look into your backend/upstream servers log files for clues using the cat command/grep command/tail command:
$ tail -f /path/to/apache.log
$ grep -i error /var/log/nginx/error.log
$ egrep -i 'error|cri' /var/log/nginx/php.error.log

Conclusion

We talked about various Nginx troubleshooting and other tips to fix upstream sent too big header while reading response header from upstream. Please see Nginx ngx_http_fastcgi_module documentation fastcgi documents here and ngx_http_proxy_module proxy docs here.

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.


ADVERTISEMENTS

Leave a Comment