Clear Memory Cache on Linux Server

By default the Linux OS has a very efficient memory management process that should be freeing any cached memory on the machine that it is being run on. However when it comes to Cached memory the Linux OS may at times decide that the Cached memory is being used and is needed which can lead to memory related issues and ultimately rob your server of any potentially free memory. To combat this you can force the Linux OS to free up and stored Cached memory.

Connect via shell using a program such as Putty
At the shell prompt type crontab -e as this will allow you to edit cron jobs for the root user.
If you are not familiar with vi (linux editor) you press "i" to insert text and once done hit "esc" and type ":wq" to save the file.
Scroll to the bottom of the cron file using the arrows key and enter the following line:
0 * * * * /root/clearcache.sh
Create a file in '/root' called 'clearcache.sh' with the following content:

#!/bin/sh
sync; echo 3 > /proc/sys/vm/drop_caches
Once you have saved this file, the job is complete!



Every hour the cron job will run this command and clear any memory cache that has built up.

An example from a test server before and after running this task.

BEFORE:

AFTER:

Note before the server was using 1.918Gb of RAM with 1.4983Gb in Cache and after running the script the server is now only using 172Mb of RAM and only 38.9Gb in Cache.

Bind Multiple IP Addresses to a Single Network Interface Card (NIC)

The network scripts are located in /etc/sysconfig/network-scripts/. Go into that directory.

cd /etc/sysconfig/network-scripts/

The file we're interested in is ifcfg-eth0, the interface for the Ethernet device. Let's assume we want to bind three additional IP's (192.168.1.111, 192.168.1.112, and 192.168.1.113) to the NIC. We need to create three alias files while ifcfg-eth0 maintains the primary IP address. This is how we'll set up the aliases to bind the IP addresses.

Adapter IP Address Type

-----------------------------------

eth0 192.168.1.110 Primary

eth0:0 192.168.1.111 Alias 1

eth0:1 192.168.1.112 Alias 2

eth0:2 192.168.1.113 Alias 3

The :X (where X is the interface number) is appended to the interface file name to create the alias. For each alias you create you assign a number sequentially. For this example we will create aliases for eth0. Make a copy of ifcfg-eth0 for the three aliases.

cp ifcfg-eth0 ifcfg-eth0:0
cp ifcfg-eth0 ifcfg-eth0:1
cp ifcfg-eth0 ifcfg-eth0:2

Take a look inside ifcfg-eth0 and review the contents.

more ifcfg-eth0

We're interested in only two lines (DEVICE and IPADDR). We'll rename the device in each file to its corresponding interface alias and change the IP's. We'll start with ifcfg-eth0:0. Open ifcfg-eth0:0 in vi and change the two lines so they have the new interface and IP address.

vi ifcfg-eth0:0

DEVICE=eth0:0

IPADDR=192.168.1.111

Save ifcfg-eth0:0 and edit the other two alias files (ifcfg-eth0:1 and ifcfg-eth0:2) so they have the new interfaces and IP addresses set (follow the table from above). Once you save all your changes you can restart the network for the changes to take effect.

service network restart

To verify all the aliases are up and running you can run ifconfig (depending on how many new IP's you set up, you can use ifconfig | more to pause the output).

ifconfig

You can also test the IP's by pinging them from a different machine. If everything is working then there should be a response back.

ping 192.168.1.111
ping 192.168.1.112
ping 192.168.1.113

Apache optimization

compressing javascript and css files with htaccess

Compressing is one of those optimizations for a website which can be easily done and still has tremendous effect. Gzip compressing your files results in less bandwidth being used and a lot less data that needs to travel all the way to the impatient client.

gzip compressing is a piece of cake when you can use htaccess. You can put the code below in your .htaccess file:



SetOutputFilter DEFLATE

< /Files>




SetOutputFilter DEFLATE

< /Files>



This will use an output filter to compress the files. All .js and .css files within the jurisdiction of this .htaccess file will be sent compressed to the client.

compressing HTML within your PHP files


Compressing is one of those optimizations for a website which can be easily done and still has tremendous effect. Gzip compressing your files results in less bandwidth being used and a lot less data that needs to travel all the way to the impatient client.

gzip compressing your PHP files is a piece of cake with Output Buffering (OB). OB buffers everything you output in your scripts, and releases it when you want it to. This neat little trick can come in handy when you want to send headers from within your HTML, since the headers are ordered to be released before the HTML when OB is used.

OB can gzip compress the HTML buffered, so you can put this above everything in your PHP file:

if (substr_count($_SERVER["HTTP_ACCEPT_ENCODING"], "gzip"))

ob_start("ob_gzhandler");

else

ob_start();


It checks whether gzip is supported, and then starts output buffering with gzip compression. To release all compressed output you can put this at the bottom of the PHP file:

ob_end_flush();


This flushes all output to the browser.

Finding the largest files and / or directories on a Linux system

Errors about disks being full is always a pain. With normal use of the system, you never know what takes up the most space, which you must if you want to clean up your system efficiently.

Unfortunately there's no command to locate your largest files or directories on a Linux system. Though piping a few commands can easily help you get the list of files and directories you want.

- du : Checking size on files or directories
- sort : Sorting lines of given data
- head : Limit output to the first part of the original output

So this is what you can enter if you want to know the top 10 of the largest files and / or directories on your Linux system. Du checks size (-a for all files and dirs), sort sorts the data it receives from du (-n for numeric sorting, -r for reversing the sort), and throws it at head, which takes the top 10 and shows it.

du -a /var | sort -n -r | head -n 10

Automatically find / delete old files on Linux

Deleting old files in Linux is often necessary. Often logs need to be periodically removed for example. Accomplishing this through bash scripts is a nuisance. Luckily there is the Find utility, it allows a few very interesting arguments, one of them is executing a command when a file is found. This argument can be used to call rm, thus, enabling you to remove what you find. Another argument Find allows can specify a time in which should be searched. This way you can delete files older than 10 days, or older than 30 minutes. A combination of these arguments can be used to do what we want.

First off, we need to find files older than, for example, 10 days.

find /var/log -mtime +10


You can also find files older than, say, 30 minutes:

find /tmp -mmin +30


Another argument Find accepts is executing commands when it finds something. You can remove files older than x days like this:

find /path/* -mtime +5 -exec rm {} \;



The " {} " represents the file found by Find, so you can feed it to rm. The " \; " ends the command that needs to be executed, which you need to unless you want errors like:

find: missing argument to `-exec`


Like i said, {} represents the file. You can do anything with a syntax like this. You can also move files around with Find:

find ~/projects/* -mtime +14 -exec mv {} ~/old_projects/ \;



Which effectively moves the files in ~/projects to ~/old_projects when their older than 14 days.

Mysql Optimize Configuration

# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html
#
# Take care to only add/remove/change a setting if you are comfortable
# doing so! For Rackspace customers, if you have any questions or
# concerns, please contact the MySQL Database Services Team. Be aware
# that some work performed by this team can involve additional billable
# fees.
#
# This file generated for host Boodhii please modify
# variables if the server is resized from 4194304kB

[mysqld]

### General
user = mysql
port = 3306
datadir = /var/lib/mysql
tmpdir = /var/lib/mysqltmp
socket = /var/lib/mysql/mysql.sock
skip-external-locking = 1

## This prevents using host-based authentication. That means users must be
## created using an ip-address (ie 'myuser'@'192.168.100.1') or must make
## use of the % wildcard (ie 'myuser'@'%'). The benefit to not using
## host-based authentication is that DNS will not impact MySQL performance.
#skip-name-resolve

## If open-files-limit is set very low, MySQL may increase on its own. Either
## way, increase this if MySQL gives 'too many open files' errors. Setting
## this above 65535 could be unwise (MySQL may crash).
open-files-limit = 20000

## Set this to change the way MySQL handles validation, data conversion, etc.
## Be careful with this setting as it can cause unexpected results and
## horribly break some applications! Note, too, that it can be set per-session
## and can be hard set in stored procedures.
#sql-mode = TRADITIONAL

#event-scheduler = 1

### Cache
thread-cache-size = 16
table-open-cache = 2048
table-definition-cache = 512

## Generally, it is unwise to set the query cache to be larger than 64-128M
## as the costs associated with maintaining the cache outweigh the performance
## gains. A far superior solution would be to implement memcached, though this
## required modifying the application, among other things.
query-cache-size = 32M
query-cache-limit = 1M

### Per-thread Buffers
sort-buffer-size = 1M
read-buffer-size = 1M
read-rnd-buffer-size = 8M
join-buffer-size = 1M

### Temp Tables
tmp-table-size = 64M
max-heap-table-size = 64M

### Networking
back-log = 100
max-connections = 200
max-connect-errors = 10000
max-allowed-packet = 16M
interactive-timeout = 600
wait-timeout = 180
net_read_timeout = 30
net_write_timeout = 30
# This value is the size of the listen queue for incoming TCP/IP connections.
back_log = 128

#### Storage Engines
## Set this to force MySQL to use a particular engine / table-type
## for new tables. This setting can still be overridden by specifying
## the engine explicitly in the CREATE TABLE statement.
#default-storage-engine = InnoDB

## Makes sure MySQL does not start if InnoDB fails to start. This helps
## prevent ugly silent failures.
innodb = FORCE

### MyISAM
## Not sure what to set this to?
## Try running a 'du -sch /var/lib/mysql/*/*.MYI'
## This will give you a good estimate on the size of all the MyISAM indexes.
## (The buffer may not need to set that high, however)
key-buffer-size = 64M
## This setting controls the size of the buffer that is allocated when
## sorting MyISAM indexes during a REPAIR TABLE or when creating indexes
## with CREATE INDEX or ALTER TABLE.
myisam-sort-buffer-size = 128M

### InnoDB
## Note: While most settings in MySQL can be set at run-time, many InnoDB
## variables cannot be set at runtime as require restarting MySQL
###
## These settings control how much RAM InnoDB will use. Generally, when using
## mostly InnoDB tables, the innodb-buffer-pool-size should be as large as
## is possible without swapping or starving other processes of RAM. The other
## two settings usually do not need to be changed, but can help for very large
## datasets.
innodb-buffer-pool-size = 16M
innodb-log-buffer-size = 4M
#innodb-additional-mem-pool-size= 20M

## This can help, but can also hinder performance. Test appropriately!
## (For SAN, O_DIRECT is almost never a good idea)
#innodb-flush-method = O_DIRECT

## innodb-file-per-table can offer quite a few advantages, but does not work
## well when using it with a very large number of tables.
## If innodb-file-per-table is used, be sure to set innodb-open-files
## appropriately (which is roughly similar to open-files-limit, but is
## exclusive to InnoDB)
#innodb-file-per-table = 1
#innodb-open-files = 300

## If you are not sure what to set this to, the following formula can offer
## up a rough idea:
## (number of cpus * number of disks * 2)
#innodb-thread-concurrency = 16

## This can increase performance for single servers as disabling this enabled
## group-commit. This is not a viable option when using binary logging or
## replication, however.
#innodb-support-xa = 0

## Be careful when changing these as they require re-generating the
## ib-logfile* files, which must be done carefully. Do not change this unless
## you are familiar with the procedure.
#innodb-log-file-size = 100M
#innodb-log-group-home-dir = /var/lib/mysql
innodb-log-files-in-group = 2

## You cannot change this without dumping out the data and re-importing it!
#innodb-data-file-path = ibdata1:2000M;ibdata2:10M:autoextend
#innodb-data-home-dir = /var/lib/mysql

### Replication

## Tired of running into replication errors due to having the same server
## id on two servers? Consider changing this variable to the Rackspace
## server number.
server-id = 1

## This sets the format used when logging to the binary log
## - ROW will force row-based logging
## - STATEMENT will force statement-based (ie pre 5.1) logging
## - MIXED wil use both, depending on the situation
## Note that this setting has implications for both replication and
## backups, so do not change this unless you know what you are doing!
#binlog-format = STATEMENT

#log-bin = /var/lib/mysqllogs/bin-log
#relay-log = /var/lib/mysqllogs/relay-log
#relay-log-space-limit = 4G
#expire-logs-days = 14

## This should be enabled on conventional MySQL slaves
#read-only = 1

## Enable this to make replication more resilient against server
## crashes and restarts, at the expense of higher I/O on the server.
#sync-binlog = 1

## This is usually only needed when setting up chained replication.
#log-slave-updates = 1

## Uncomment the following when enabling multi-master replication
## Do NOT uncomment these unless you know exactly what you are doing!
#auto-increment-offset = 1
#auto-increment-increment = 2

### Logging
## This option determines the destination for general query log and slow query log output.
## The option value can be given as one or more of the words TABLE, FILE, or NONE.
## NOTE: Table logging takes away 50% of performance and thus is not recommended
## http://bugs.mysql.com/bug.php?id=30414
## In addition, you cannot backup the contents of these tables properly
## (mysqldump skips these tables by default since they cannot be locked)
#log-output = FILE
#slow-query-log = 1
#slow-query-log-file = /var/lib/mysqllogs/slow-log
#long-query-time = 2
#log-queries-not-using-indexes = 1

[mysqld-safe]
log-error = /var/log/mysqld.log

[mysqldump]
max-allowed-packet = 16M

# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/sysconfig/mysqld-config/


http://everythingmysql.ning.com/profiles/blogs/using-tmpfs-for-mysqls-tmpdir

I would like to talk about not "why" MySQL does this but how to speed up the performance when MySQL internally or users create temporary tables to disk. A great solution is TMPFS, a quick how to is as follows:

-- Before you start
1. Make sure you allocate enough space to TMPFS
-- 2GB is usually safe but if you are using larger data sets with inefficient queries then there are far worse performance issues to deal with.

-- The safe way to implement TMPFS for MySQL
shell> mkdir /tmp/mysqltmp
shell> chown mysql:mysql /tmp/mysqltmp
shell> id mysql
##NOTE: make sure you get the uid and gid for mysql
shell> vi /etc/fstab
## make sure this in in your fstab
tmpfs /tmp/mysqltmp tmpfs rw,uid=25,gid=26,size=2G,nr_inodes=10k,mode=0700 0 0
shell> mount /tmp/mysqltmp
shell> vi /etc/my.cnf #or the mysql config file for your server
## NOTE: inside the file add the following under [mysqld]
tmpdir=/tmp/mysqltmp/
shell> service mysql restart

How to Set time & timezone in mysql

SELECT CURRENT_TIMESTAMP;

SET GLOBAL time_zone = '-5:00';

 

Apache Optimize Configuration

ServerRoot "/etc/httpd"
PidFile run/httpd.pid
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5


StartServers 4
MinSpareServers 9
MaxSpareServers 18
ServerLimit 279
MaxClients 279
MaxRequestsPerChild 1000



StartServers 4
MaxClients 1024
MinSpareThreads 64
MaxSpareThreads 192
ThreadsPerChild 64
MaxRequestsPerChild 0


Include /etc/httpd/ports.conf
ServerAdmin root@localhost
UseCanonicalName Off
DocumentRoot "/var/www/html"

Options -Indexes FollowSymLinks
AllowOverride All
DirectoryIndex index.php default.php index.html index.htm index.shtml index.php4 index.php3 index.phtml index.cgi
AccessFileName .htaccess
TypesConfig /etc/mime.types
HostnameLookups Off
ErrorLog /var/log/httpd/error_log
Redirect permanent /foo http://www.example.com/bar
Include vhost.d/*.conf

Securites.conf

ServerTokens Prod
ServerSignature Off
TraceEnable Off

ports.conf

Listen 80
NameVirtualHost *:80

Listen 443
NameVirtualHost *:443

vhost/servername.conf


ServerName example.com
ServerAlias www.example.com
DocumentRoot /var/www/vhosts/example.com

Options Indexes FollowSymLinks MultiViews
AllowOverride All
DirectoryIndex index.php default.php index.html index.htm index.shtml index.php4 index.php3 index.phtml index.cgi


CustomLog /var/log/httpd/example.com-access.log combined
ErrorLog /var/log/httpd/example.com-error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn





ServerName example.com
DocumentRoot /var/www/vhosts/example.com

Options Indexes FollowSymLinks MultiViews
AllowOverride All
DirectoryIndex index.php default.php index.html index.htm index.shtml index.php4 index.php3 index.phtml index.cgi


CustomLog /var/log/httpd/example.com-ssl-access.log combined
ErrorLog /var/log/httpd/example.com-ssl-error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key


SSLOptions +StdEnvVars


BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown