#StackBounty: #cron #postfix #nginx #smtp #stability A utility to email myself if my site is down

Bounty: 200

I use Ubuntu 16.04 with Nginx and a few WordPress sites. Sometimes I don’t visit a site for a long time (>=1 month) and it might be that the site is down.

I’m looking for a small utility that will email my Gmail account, if one of my Nginx-WordPress sites is down (without mentioning a reason).

Approaches considered so far

1. Creating a tool from scratch

  1. Creating the whole non-default configuration for my SMTP server.
  2. Adding anc configuring DNS recors at the hosting providers DNS management tool.
  3. Adding a weekly cron task with curl -l -L on each domain and save it’s output into a file.
  4. Adding a weekly cron task of say one hour later, to check each file and email myself if the status code isn’t 200.

This might seem simple, but is actually quite complex (though not necessarily complicated), and it also might be a bit fragile. A dedicated, communal, maintained utility might be better for me.

2. Third party tools

I don’t want to use some grandiose, third-party network-monitoring service like Nagios, Icinga, Zabbix, Shinken, etc, and they all seem an overkill per this particular cause.

3. Postfix add-on

I’ve already installed Postfix with the internet-site configuration so that tool might utilize Postfix. I just use the Postfix defaults, some default conf I could add on top of internet-site, maybe without adding and configuring DNS records.

A utility which is an interactive program to re-configure Postfix might ease my pain; I wouldn’t have to fill my Ubuntu-Nginx-WordPress-Environment installation-script with much SMTP configuration data. Maybe I’ll just have to set some DNS records after that, and that’s it. Anything that would ease the process this way or another is also an option for me.

4. Handling the spam filter

Even if Gmail would mistakenly move my first email (or the first series of email) to spam, I could put it into a whitelist.


My question

Is there a utility I could use to have this behavior?


Get this bounty!!!

#StackBounty: #8 #nginx Does the Nginx default block suffice?

Bounty: 200

The following conf file is my default Nginx conf (it’s basically the one shipped with Nginx 1.10.3 in Ubuntu).

server {
    # listen, root, index, server_name, locations;
    listen 80 default_server;
    listen [::]:80 default_server;
    root /var/www/html;
    index index.php index.html index.htm index.nginx-debian.html;
    server_name _;

    location / {
        try_files $uri $uri/ =404;
    }
    location ~ .php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
    }
}

This is my individual app Nginx configuration.

server {
    root ${drt}/${domain}/;
    server_name ${domain} www.${domain};
    location ~* .(jpg|jpeg|png|gif|ico|css|js|ttf|woff|pdf)$ { expires 365d; }
}

Are these Nginx configuration files enough for basic, all-default Drupal usage (i.e. all core, no module customization, and up to 5 typical third-party modules)?

Enough in the sense of getting Drupal 8.4.x or later to work, with basic security.


Get this bounty!!!

#StackBounty: #dns #nginx #webserver #dnsmasq #spoofing NGINX Redirect to Domain Name spoofed locally with DNSMasq

Bounty: 150

I have a Linux VPS serving data on the internet that has a legitimate Domain name and SSL Certificate ( from GoDaddy.com ). I will refere to this server as “www.myserver.com“. I also have a local Linux machine ( on my own LAN ) that I want to use to DNS spoof my internet facing Domain name ( www.myserver.com ) to it’s own NGINX webserver running on that local machine.

I setup DNSMasq on the local machine to spoof that domain to it’s local 192.x address and I verified from another machine on the LAN that dig reports the local address.

Local server dnsmaq spoof mapping:

cat /etc/dnsmasq.d/spoof.hosts 
192.168.1.142 www.myserver.com myserver.com

Separate machine on LAN shows that spoofed mapping should work:

dig +short @192.168.1.142 myserver.com
>> 192.168.1.142

My dnsmasq.conf:

server=127.0.0.1
listen-address=127.0.0.1
listen-address=192.168.1.142
no-dhcp-interface=
no-hosts
addn-hosts=/etc/dnsmasq.d/spoof.hosts

My spoof.hosts:

192.168.1.142 www.myserver.com myserver.com

On the local server, I configured NGINX with resolver to look to localhost for DNS as shown here:

http {

    access_log off;
    include mime.types;

    default_type html;

    sendfile        on;

    keepalive_requests 50;
    keepalive_timeout  75s;

    reset_timedout_connection on;

    server_tokens off;

    server {

       listen 8080 default_server;

       resolver 127.0.0.1 valid=10s;

       location / {
           return 302 http://myserver.com/;
       }

    }

    server {

      listen       80;
      server_name *.myserver.com;

      // Various Endpoints

    }
}

The problem is that when I visit my local machine 192.168.1.131:8080, it redirects to my actual internet facing machine – the real domain name on the internet.

I want it to redirect to the local spoofed DNS. What am I doing wrong? How can I accomplish this? Thank you.

UPDATE: I’ve tried this as well but no luck:

http {
    access_log off;
    include mime.types;

    default_type html;

    sendfile        on;

    keepalive_requests 50;
    keepalive_timeout  75s;

    reset_timedout_connection on;

    server_tokens off;

    server {

          listen 80 default_server;
          server_name _;
      resolver 127.0.0.1;
          return 301 https://myserver.com/$request_uri;   

    }

    server {

        listen       443;
    server_name *.myserver.com;

    ssl on;
    ssl_certificate /etc/nginx/ssl/1e17e6d8f94cc4ee.crt;
        ssl_certificate_key /etc/nginx/ssl/example.com.key;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 
        ssl_prefer_server_ciphers on;
        ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';

    ...

    }
}


Get this bounty!!!

#StackBounty: #django #caching #nginx logout not working, caching on nginx, how to allow logout?

Bounty: 100

I have everything cached, if I logged into my account, you will not be able to log out any more) how do you get out when you quit? i need to know how to delete cookies and session! when i’ll logout!

P.S. if i’ll disable caching on nginx level, everything works fine,
problem in nginx

nginx conf

    gzip on;
    gzip_disable "msie6";

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    proxy_connect_timeout 5;
    proxy_send_timeout 10;
    proxy_read_timeout 10;
    proxy_buffering on;
    proxy_buffer_size 16k;
    proxy_buffers 24 16k;
    proxy_busy_buffers_size 64k;
    proxy_temp_file_write_size 64k;

    proxy_temp_path /tmp/nginx/proxy_temp;
    add_header X-Cache-Status $upstream_cache_status;
    proxy_cache_path /tmp/nginx/cache levels=1:2 keys_zone=first_zone:100m;
    proxy_cache one;
    proxy_cache_valid any 30d;
    proxy_cache_key $scheme$proxy_host$request_uri$cookie_US;

server conf

upstream some site {
  server unix:/webapps/some/run/gunicorn.sock fail_timeout=0;
}

server {
    listen   80;
    server_name server name;
    expires 7d;
    client_max_body_size 4G;

    access_log /webapps/some/logs/nginx-access.log;
    error_log /webapps/some/logs/nginx-error.log;
    error_log /webapps/some/logs/nginx-crit-error.log crit;
    error_log /webapps/some/logs/nginx-debug.log debug; 
    location /static/ {
        alias   /webapps/some/static/;
    }

    location /media/ {
        alias   /webapps/some/media/;
    }
    location ~* ^(?!/media).*.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
        root root_path;
        expires 7d;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        access_log off;
    }    
    location ~* ^(?!/static).*.(?:css|js|html)$ {
        root root_path;
        expires 7d;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        access_log off;
    }     

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_cache one;
        proxy_cache_min_uses 1;
        proxy_cache_use_stale error timeout;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        if (!-f $request_filename) {
            proxy_pass http://some;
            break;
        }
    }
    error_page 404 /404.html;
    location = /error_404.html {
        root /webapps/some/src/templates;
    }

    error_page  500 502 503 504 /500.html;
    location = /error_500.html {
        root /webapps/some/src/templates;
    }
}


Get this bounty!!!

#StackBounty: #nginx #heroku #lazy-loading #nginx-reverse-proxy Can web applications start on demand using nginx (like heroku does)

Bounty: 50

I want to build an application server to host multiple applications, but keeping those applications sleeping most of the time and starting them on demand, like heroku router does for free applications.

Does I need to add a proxy application that start the applications or can nginx be configured to handle that in some way?

I thought about adding a node application that start the application on-demand and proxy the request there, but I’m afraid it might add much time as it will need to proxy every request and also I might loose the first request as the application might not be ready to answer that.

I’ve also thought that I could use the ngx_http_auth_request module to start the application, but I’m not sure if that could work.

The idea is that I don’t have a list of all the applications I will have running, as new applications can be installed anytime.


Get this bounty!!!

#StackBounty: #nginx #cache #django nginx caching work wrong, nginx cache auth system, logout not work (on DJANGO), how to fix?

Bounty: 50

nginx is caching everything, if I login to the system, then I can no longer exit it until the caching expires, since I’m Logout from the account, i need to know how to delete cookies and session!

by default, Django itself removes cookies and sessions when exiting,
using the standard method to exit the account from the developers
django, I use it, if you disable caching at nginx, then everything
works fine!

nginx conf “/etc/nginx/nginx.conf”

##
# Logging Settings
##

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

##
# Gzip Settings
##

gzip on;
gzip_disable "msie6";

gzip_vary on;
gzip_proxied any;
    gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

proxy_connect_timeout 5;
proxy_send_timeout 10;
proxy_read_timeout 10;

proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 24 16k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;

proxy_temp_path /tmp/nginx/proxy_temp;
add_header X-Cache-Status $upstream_cache_status;
    proxy_cache_path /tmp/nginx/cache levels=1:2 keys_zone=one:100m;
proxy_cache_path /tmp/nginx/cache2 levels=1:2 keys_zone=two:100m;
proxy_cache one;
proxy_cache_valid any 30d;
proxy_cache_key $scheme$proxy_host$request_uri$cookie_US;

my server conf

upstream theband {
  # fail_timeout=0 means we always retry an upstream even if it failed
  # to return a good HTTP response (in case the Unicorn master nukes a
  # ssingle worker for timing out).

  server unix:/webapps/theband/run/gunicorn.sock fail_timeout=0;
}

server {

    listen   80;
    server_name 207.154.232.99;
    expires 35d;
    client_max_body_size 4G;

    access_log /webapps/theband/logs/nginx-access.log;
    error_log /webapps/theband/logs/nginx-error.log;
    error_log /webapps/theband/logs/nginx-crit-error.log crit;
    error_log /webapps/theband/logs/nginx-debug.log debug; 
    location /static/ {
        alias   /webapps/theband/static/;
    }

    location /media/ {
        alias   /webapps/theband/media/;
    }
    location ~* ^(?!/media).*.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
        alias /tmp/nginx/trash/trash_media;
        expires 35d;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        access_log off;
    }    
    location ~* ^(?!/static).*.(?:css|js|html)$ {
    root /tmp/nginx/trash/trash_static;
        expires 35d;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        access_log off;
    }     

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_cache one;
        proxy_cache_min_uses 1;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;

        # an HTTP header important enough to have its own Wikipedia entry:
        #   http://en.wikipedia.org/wiki/X-Forwarded-For
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # enable this if and only if you use HTTPS, this helps Rack
        # set the proper protocol for doing redirects:
        # proxy_set_header X-Forwarded-Proto https;

        # pass the Host: header from the client right along so redirects
        # can be set properly within the Rack application
        proxy_set_header Host $http_host;

        # we don't want nginx trying to do something clever with
        # redirects, we set the Host: header above already.
        proxy_redirect off;

        # set "proxy_buffering off" *only* for Rainbows! when doing
        # Comet/long-poll stuff.  It's also safe to set if you're
        # using only serving fast clients with Unicorn + nginx.
        # Otherwise you _want_ nginx to buffer responses to slow
        # clients, really.
        #proxy_buffering off;

        # Try to serve static files from nginx, no point in making an
        # *application* server like Unicorn/Rainbows! serve static files.
        if (!-f $request_filename) {
            proxy_pass http://theband;
            break;
        }
    }
    error_page 404 /error_404.html;
    location = /error_404.html {
        root /webapps/theband/src/templates;
    }

    # Error pages
    error_page  500 502 503 504 /error_500.html;
    location = /error_500.html {
        root /webapps/theband/src/templates;
    }
}


Get this bounty!!!

#StackBounty: #nginx #encryption #configuration Encode and decode pathname in nginx

Bounty: 50

Normally files can be accessed at:

http://example.com/cats/cat1.zip

I want to encode/encrypt the pathname (/cats/cat1.zip) so that the link is not normally accessible but accessible after the pathname is encrypted/encoded:

http://example.com/Y2F0cy9jYXQxLnppcAo=

I’m using base64 encoding above for simplicity but would prefer encryption. How do I do about doing this? Do I have to write a custom module?


Get this bounty!!!

#StackBounty: #nginx #php #php-fpm Why doesn't the query string appear in PHP-FPM status page?

Bounty: 100

I’ve enable the PHP-FPM status page and reconfigured nginx to display it. The website in question is a drupal website where the URL gets rewritten such that all non-existing files get their URL rewritten to be /index.php?q=<whatever> If I access the status page itself with ?full I can see that query string show up in the process list but when I navigate to a page on the site I would expect to see something similar to the above but instead get:

pool:                 www
process manager:      dynamic
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
accepted conn:        26
listen queue:         0
max listen queue:     0
listen queue len:     0
idle processes:       2
active processes:     1
total processes:      3
max active processes: 2
max children reached: 0
slow requests:        0

************************
pid:                  65
state:                Idle
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
requests:             9
request duration:     347524
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/html/index.php
last request cpu:     92.08
last request memory:  10485760

************************
pid:                  66
state:                Running
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
requests:             9
request duration:     211
request method:       GET
request URI:          /status?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

************************
pid:                  8924
state:                Idle
start time:           14/Dec/2017:21:41:04 -0500
start since:          180
requests:             8
request duration:     349676
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/html/index.php
last request cpu:     85.79
last request memory:  35651584

Is there a way to get the query string to survive the re-write or is there another tool for figuring out what PHP request specifically might be consuming CPU time? I’m basically trying to be able to track a CPU-consuming PID to a particular request if I need to do so for troubleshooting purposes.


Get this bounty!!!

#StackBounty: #nginx #php #php-fpm Why doesn't the query string appear in PHP-FPM status page?

Bounty: 100

I’ve enable the PHP-FPM status page and reconfigured nginx to display it. The website in question is a drupal website where the URL gets rewritten such that all non-existing files get their URL rewritten to be /index.php?q=<whatever> If I access the status page itself with ?full I can see that query string show up in the process list but when I navigate to a page on the site I would expect to see something similar to the above but instead get:

pool:                 www
process manager:      dynamic
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
accepted conn:        26
listen queue:         0
max listen queue:     0
listen queue len:     0
idle processes:       2
active processes:     1
total processes:      3
max active processes: 2
max children reached: 0
slow requests:        0

************************
pid:                  65
state:                Idle
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
requests:             9
request duration:     347524
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/html/index.php
last request cpu:     92.08
last request memory:  10485760

************************
pid:                  66
state:                Running
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
requests:             9
request duration:     211
request method:       GET
request URI:          /status?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

************************
pid:                  8924
state:                Idle
start time:           14/Dec/2017:21:41:04 -0500
start since:          180
requests:             8
request duration:     349676
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/html/index.php
last request cpu:     85.79
last request memory:  35651584

Is there a way to get the query string to survive the re-write or is there another tool for figuring out what PHP request specifically might be consuming CPU time? I’m basically trying to be able to track a CPU-consuming PID to a particular request if I need to do so for troubleshooting purposes.


Get this bounty!!!

#StackBounty: #nginx #php #php-fpm Why doesn't the query string appear in PHP-FPM status page?

Bounty: 100

I’ve enable the PHP-FPM status page and reconfigured nginx to display it. The website in question is a drupal website where the URL gets rewritten such that all non-existing files get their URL rewritten to be /index.php?q=<whatever> If I access the status page itself with ?full I can see that query string show up in the process list but when I navigate to a page on the site I would expect to see something similar to the above but instead get:

pool:                 www
process manager:      dynamic
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
accepted conn:        26
listen queue:         0
max listen queue:     0
listen queue len:     0
idle processes:       2
active processes:     1
total processes:      3
max active processes: 2
max children reached: 0
slow requests:        0

************************
pid:                  65
state:                Idle
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
requests:             9
request duration:     347524
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/html/index.php
last request cpu:     92.08
last request memory:  10485760

************************
pid:                  66
state:                Running
start time:           14/Dec/2017:17:06:53 -0500
start since:          16631
requests:             9
request duration:     211
request method:       GET
request URI:          /status?full
content length:       0
user:                 -
script:               -
last request cpu:     0.00
last request memory:  0

************************
pid:                  8924
state:                Idle
start time:           14/Dec/2017:21:41:04 -0500
start since:          180
requests:             8
request duration:     349676
request method:       GET
request URI:          /index.php
content length:       0
user:                 -
script:               /var/www/html/index.php
last request cpu:     85.79
last request memory:  35651584

Is there a way to get the query string to survive the re-write or is there another tool for figuring out what PHP request specifically might be consuming CPU time? I’m basically trying to be able to track a CPU-consuming PID to a particular request if I need to do so for troubleshooting purposes.


Get this bounty!!!