Apache optimization

compressing javascript and css files with htaccess

Compressing is one of those optimizations for a website which can be easily done and still has tremendous effect. Gzip compressing your files results in less bandwidth being used and a lot less data that needs to travel all the way to the impatient client.

gzip compressing is a piece of cake when you can use htaccess. You can put the code below in your .htaccess file:



SetOutputFilter DEFLATE

< /Files>




SetOutputFilter DEFLATE

< /Files>



This will use an output filter to compress the files. All .js and .css files within the jurisdiction of this .htaccess file will be sent compressed to the client.

compressing HTML within your PHP files


Compressing is one of those optimizations for a website which can be easily done and still has tremendous effect. Gzip compressing your files results in less bandwidth being used and a lot less data that needs to travel all the way to the impatient client.

gzip compressing your PHP files is a piece of cake with Output Buffering (OB). OB buffers everything you output in your scripts, and releases it when you want it to. This neat little trick can come in handy when you want to send headers from within your HTML, since the headers are ordered to be released before the HTML when OB is used.

OB can gzip compress the HTML buffered, so you can put this above everything in your PHP file:

if (substr_count($_SERVER["HTTP_ACCEPT_ENCODING"], "gzip"))

ob_start("ob_gzhandler");

else

ob_start();


It checks whether gzip is supported, and then starts output buffering with gzip compression. To release all compressed output you can put this at the bottom of the PHP file:

ob_end_flush();


This flushes all output to the browser.

Finding the largest files and / or directories on a Linux system

Errors about disks being full is always a pain. With normal use of the system, you never know what takes up the most space, which you must if you want to clean up your system efficiently.

Unfortunately there's no command to locate your largest files or directories on a Linux system. Though piping a few commands can easily help you get the list of files and directories you want.

- du : Checking size on files or directories
- sort : Sorting lines of given data
- head : Limit output to the first part of the original output

So this is what you can enter if you want to know the top 10 of the largest files and / or directories on your Linux system. Du checks size (-a for all files and dirs), sort sorts the data it receives from du (-n for numeric sorting, -r for reversing the sort), and throws it at head, which takes the top 10 and shows it.

du -a /var | sort -n -r | head -n 10

Automatically find / delete old files on Linux

Deleting old files in Linux is often necessary. Often logs need to be periodically removed for example. Accomplishing this through bash scripts is a nuisance. Luckily there is the Find utility, it allows a few very interesting arguments, one of them is executing a command when a file is found. This argument can be used to call rm, thus, enabling you to remove what you find. Another argument Find allows can specify a time in which should be searched. This way you can delete files older than 10 days, or older than 30 minutes. A combination of these arguments can be used to do what we want.

First off, we need to find files older than, for example, 10 days.

find /var/log -mtime +10


You can also find files older than, say, 30 minutes:

find /tmp -mmin +30


Another argument Find accepts is executing commands when it finds something. You can remove files older than x days like this:

find /path/* -mtime +5 -exec rm {} \;



The " {} " represents the file found by Find, so you can feed it to rm. The " \; " ends the command that needs to be executed, which you need to unless you want errors like:

find: missing argument to `-exec`


Like i said, {} represents the file. You can do anything with a syntax like this. You can also move files around with Find:

find ~/projects/* -mtime +14 -exec mv {} ~/old_projects/ \;



Which effectively moves the files in ~/projects to ~/old_projects when their older than 14 days.

Mysql Optimize Configuration

# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html
#
# Take care to only add/remove/change a setting if you are comfortable
# doing so! For Rackspace customers, if you have any questions or
# concerns, please contact the MySQL Database Services Team. Be aware
# that some work performed by this team can involve additional billable
# fees.
#
# This file generated for host Boodhii please modify
# variables if the server is resized from 4194304kB

[mysqld]

### General
user = mysql
port = 3306
datadir = /var/lib/mysql
tmpdir = /var/lib/mysqltmp
socket = /var/lib/mysql/mysql.sock
skip-external-locking = 1

## This prevents using host-based authentication. That means users must be
## created using an ip-address (ie 'myuser'@'192.168.100.1') or must make
## use of the % wildcard (ie 'myuser'@'%'). The benefit to not using
## host-based authentication is that DNS will not impact MySQL performance.
#skip-name-resolve

## If open-files-limit is set very low, MySQL may increase on its own. Either
## way, increase this if MySQL gives 'too many open files' errors. Setting
## this above 65535 could be unwise (MySQL may crash).
open-files-limit = 20000

## Set this to change the way MySQL handles validation, data conversion, etc.
## Be careful with this setting as it can cause unexpected results and
## horribly break some applications! Note, too, that it can be set per-session
## and can be hard set in stored procedures.
#sql-mode = TRADITIONAL

#event-scheduler = 1

### Cache
thread-cache-size = 16
table-open-cache = 2048
table-definition-cache = 512

## Generally, it is unwise to set the query cache to be larger than 64-128M
## as the costs associated with maintaining the cache outweigh the performance
## gains. A far superior solution would be to implement memcached, though this
## required modifying the application, among other things.
query-cache-size = 32M
query-cache-limit = 1M

### Per-thread Buffers
sort-buffer-size = 1M
read-buffer-size = 1M
read-rnd-buffer-size = 8M
join-buffer-size = 1M

### Temp Tables
tmp-table-size = 64M
max-heap-table-size = 64M

### Networking
back-log = 100
max-connections = 200
max-connect-errors = 10000
max-allowed-packet = 16M
interactive-timeout = 600
wait-timeout = 180
net_read_timeout = 30
net_write_timeout = 30
# This value is the size of the listen queue for incoming TCP/IP connections.
back_log = 128

#### Storage Engines
## Set this to force MySQL to use a particular engine / table-type
## for new tables. This setting can still be overridden by specifying
## the engine explicitly in the CREATE TABLE statement.
#default-storage-engine = InnoDB

## Makes sure MySQL does not start if InnoDB fails to start. This helps
## prevent ugly silent failures.
innodb = FORCE

### MyISAM
## Not sure what to set this to?
## Try running a 'du -sch /var/lib/mysql/*/*.MYI'
## This will give you a good estimate on the size of all the MyISAM indexes.
## (The buffer may not need to set that high, however)
key-buffer-size = 64M
## This setting controls the size of the buffer that is allocated when
## sorting MyISAM indexes during a REPAIR TABLE or when creating indexes
## with CREATE INDEX or ALTER TABLE.
myisam-sort-buffer-size = 128M

### InnoDB
## Note: While most settings in MySQL can be set at run-time, many InnoDB
## variables cannot be set at runtime as require restarting MySQL
###
## These settings control how much RAM InnoDB will use. Generally, when using
## mostly InnoDB tables, the innodb-buffer-pool-size should be as large as
## is possible without swapping or starving other processes of RAM. The other
## two settings usually do not need to be changed, but can help for very large
## datasets.
innodb-buffer-pool-size = 16M
innodb-log-buffer-size = 4M
#innodb-additional-mem-pool-size= 20M

## This can help, but can also hinder performance. Test appropriately!
## (For SAN, O_DIRECT is almost never a good idea)
#innodb-flush-method = O_DIRECT

## innodb-file-per-table can offer quite a few advantages, but does not work
## well when using it with a very large number of tables.
## If innodb-file-per-table is used, be sure to set innodb-open-files
## appropriately (which is roughly similar to open-files-limit, but is
## exclusive to InnoDB)
#innodb-file-per-table = 1
#innodb-open-files = 300

## If you are not sure what to set this to, the following formula can offer
## up a rough idea:
## (number of cpus * number of disks * 2)
#innodb-thread-concurrency = 16

## This can increase performance for single servers as disabling this enabled
## group-commit. This is not a viable option when using binary logging or
## replication, however.
#innodb-support-xa = 0

## Be careful when changing these as they require re-generating the
## ib-logfile* files, which must be done carefully. Do not change this unless
## you are familiar with the procedure.
#innodb-log-file-size = 100M
#innodb-log-group-home-dir = /var/lib/mysql
innodb-log-files-in-group = 2

## You cannot change this without dumping out the data and re-importing it!
#innodb-data-file-path = ibdata1:2000M;ibdata2:10M:autoextend
#innodb-data-home-dir = /var/lib/mysql

### Replication

## Tired of running into replication errors due to having the same server
## id on two servers? Consider changing this variable to the Rackspace
## server number.
server-id = 1

## This sets the format used when logging to the binary log
## - ROW will force row-based logging
## - STATEMENT will force statement-based (ie pre 5.1) logging
## - MIXED wil use both, depending on the situation
## Note that this setting has implications for both replication and
## backups, so do not change this unless you know what you are doing!
#binlog-format = STATEMENT

#log-bin = /var/lib/mysqllogs/bin-log
#relay-log = /var/lib/mysqllogs/relay-log
#relay-log-space-limit = 4G
#expire-logs-days = 14

## This should be enabled on conventional MySQL slaves
#read-only = 1

## Enable this to make replication more resilient against server
## crashes and restarts, at the expense of higher I/O on the server.
#sync-binlog = 1

## This is usually only needed when setting up chained replication.
#log-slave-updates = 1

## Uncomment the following when enabling multi-master replication
## Do NOT uncomment these unless you know exactly what you are doing!
#auto-increment-offset = 1
#auto-increment-increment = 2

### Logging
## This option determines the destination for general query log and slow query log output.
## The option value can be given as one or more of the words TABLE, FILE, or NONE.
## NOTE: Table logging takes away 50% of performance and thus is not recommended
## http://bugs.mysql.com/bug.php?id=30414
## In addition, you cannot backup the contents of these tables properly
## (mysqldump skips these tables by default since they cannot be locked)
#log-output = FILE
#slow-query-log = 1
#slow-query-log-file = /var/lib/mysqllogs/slow-log
#long-query-time = 2
#log-queries-not-using-indexes = 1

[mysqld-safe]
log-error = /var/log/mysqld.log

[mysqldump]
max-allowed-packet = 16M

# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/sysconfig/mysqld-config/


http://everythingmysql.ning.com/profiles/blogs/using-tmpfs-for-mysqls-tmpdir

I would like to talk about not "why" MySQL does this but how to speed up the performance when MySQL internally or users create temporary tables to disk. A great solution is TMPFS, a quick how to is as follows:

-- Before you start
1. Make sure you allocate enough space to TMPFS
-- 2GB is usually safe but if you are using larger data sets with inefficient queries then there are far worse performance issues to deal with.

-- The safe way to implement TMPFS for MySQL
shell> mkdir /tmp/mysqltmp
shell> chown mysql:mysql /tmp/mysqltmp
shell> id mysql
##NOTE: make sure you get the uid and gid for mysql
shell> vi /etc/fstab
## make sure this in in your fstab
tmpfs /tmp/mysqltmp tmpfs rw,uid=25,gid=26,size=2G,nr_inodes=10k,mode=0700 0 0
shell> mount /tmp/mysqltmp
shell> vi /etc/my.cnf #or the mysql config file for your server
## NOTE: inside the file add the following under [mysqld]
tmpdir=/tmp/mysqltmp/
shell> service mysql restart

How to Set time & timezone in mysql

SELECT CURRENT_TIMESTAMP;

SET GLOBAL time_zone = '-5:00';

 

Apache Optimize Configuration

ServerRoot "/etc/httpd"
PidFile run/httpd.pid
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5


StartServers 4
MinSpareServers 9
MaxSpareServers 18
ServerLimit 279
MaxClients 279
MaxRequestsPerChild 1000



StartServers 4
MaxClients 1024
MinSpareThreads 64
MaxSpareThreads 192
ThreadsPerChild 64
MaxRequestsPerChild 0


Include /etc/httpd/ports.conf
ServerAdmin root@localhost
UseCanonicalName Off
DocumentRoot "/var/www/html"

Options -Indexes FollowSymLinks
AllowOverride All
DirectoryIndex index.php default.php index.html index.htm index.shtml index.php4 index.php3 index.phtml index.cgi
AccessFileName .htaccess
TypesConfig /etc/mime.types
HostnameLookups Off
ErrorLog /var/log/httpd/error_log
Redirect permanent /foo http://www.example.com/bar
Include vhost.d/*.conf

Securites.conf

ServerTokens Prod
ServerSignature Off
TraceEnable Off

ports.conf

Listen 80
NameVirtualHost *:80

Listen 443
NameVirtualHost *:443

vhost/servername.conf


ServerName example.com
ServerAlias www.example.com
DocumentRoot /var/www/vhosts/example.com

Options Indexes FollowSymLinks MultiViews
AllowOverride All
DirectoryIndex index.php default.php index.html index.htm index.shtml index.php4 index.php3 index.phtml index.cgi


CustomLog /var/log/httpd/example.com-access.log combined
ErrorLog /var/log/httpd/example.com-error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn





ServerName example.com
DocumentRoot /var/www/vhosts/example.com

Options Indexes FollowSymLinks MultiViews
AllowOverride All
DirectoryIndex index.php default.php index.html index.htm index.shtml index.php4 index.php3 index.phtml index.cgi


CustomLog /var/log/httpd/example.com-ssl-access.log combined
ErrorLog /var/log/httpd/example.com-ssl-error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key


SSLOptions +StdEnvVars


BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

Hardisk Partion Create

Q. I've installed a new 250GB SATA hard disk on our office CentOS Linux server. How do I format a hard disk under Linux operating system from a shell prompt?



A.. There are total 4 steps involved for hard disk upgrade and installation procedure:

Step #1 : Partition the new disk using fdisk command

Following command will list all detected hard disks:

# fdisk -l | grep '^Disk'

Output:

Disk /dev/sda: 251.0 GB, 251000193024 bytes
Disk /dev/sdb: 251.0 GB, 251000193024 bytes

A device name refers to the entire hard disk. For more information see Linux partition naming convention and IDE drive mappings.

To partition the disk - /dev/sdb, enter:

# fdisk /dev/sdb

The basic fdisk commands you need are:

• m - print help


• p - print the partition table


• n - create a new partition


• d - delete a partition


• q - quit without saving changes


• w - write the new partition table and exit

Step#2 : Format the new disk using mkfs.ext3 command

To format Linux partitions using ext2fs on the new disk:

# mkfs.ext3 /dev/sdb1

Step#3 : Mount the new disk using mount command

First create a mount point /disk1 and use mount command to mount /dev/sdb1, enter:

# mkdir /disk1

# mount /dev/sdb1 /disk1

# df -H

Step#4 : Update /etc/fstab file

Open /etc/fstab file, enter:

# vi /etc/fstab

Append as follows:

/dev/sdb1               /disk1           ext3    defaults        1 2

Save and close the file.

Task: Label the partition

You can label the partition using e2label. For example, if you want to label the new partition /backup, enter

# e2label /dev/sdb1 /backup

You can use label name insted of partition name to mount disk using /etc/fstab:

LABEL=/backup /disk1 ext3 defaults 1 2

Featured Articles:

• 20 Linux System Monitoring Tools Every SysAdmin Should Know


• 20 Linux Server Hardening Security Tips


• My 10 UNIX Command Line Mistakes


• The Novice Guide To Buying A Linux Laptop


• 10 Greatest Open Source Software Of 2009


• Top 5 Email Client For Linux, Mac OS X, and Windows Users


• Top 20 OpenSSH Server Best Security Practices


• Top 10 Open Source Web-Based Project Management Software


• Top 5 Linux Video Editor Software

Difference between TCP and UDP

TCP	UDP
Reliability: TCP is connection-oriented protocol. When a file or message send it will get delivered unless connections fails. If connection lost, the server will request the lost part. There is no corruption while transferring a message.	Reliability: UDP is connectionless protocol. When you a send a data or message, you don't know if it'll get there, it could get lost on the way. There may be corruption while transferring a message.
Ordered: If you send two messages along a connection, one after the other, you know the first message will get there first. You don't have to worry about data arriving in the wrong order.	Ordered: If you send two messages out, you don't know what order they'll arrive in i.e. no ordered
Heavyweight: - when the low level parts of the TCP "stream" arrive in the wrong order, resend requests have to be sent, and all the out of sequence parts have to be put back together, so requires a bit of work to piece together.	Lightweight: No ordering of messages, no tracking connections, etc. It's just fire and forget! This means it's a lot quicker, and the network card / OS have to do very little work to translate the data back from the packets.
Streaming: Data is read as a "stream," with nothing distinguishing where one packet ends and another begins. There may be multiple packets per read call.	Datagrams: Packets are sent individually and are guaranteed to be whole if they arrive. One packet per one read call.
Examples: World Wide Web (Apache TCP port 80), e-mail (SMTP TCP port 25 Postfix MTA), File Transfer Protocol (FTP port 21) and Secure Shell (OpenSSH port 22) etc.	Examples: Domain Name System (DNS UDP port 53), streaming media applications such as IPTV or movies, Voice over IP (VoIP), Trivial File Transfer Protocol (TFTP) and online multiplayer games etc