Getting rid of control characters from mail

http://www.math.iitb.ac.in/~srg/LaTeX/howto.html

use global substitution
:g/cntrl-v cntrl-m/s// /g

Checking memory

try sar(system activity reporting), it is a good alternative to top. My rhel box was having i/o wait which I could find using sar.

sar 1 200
Other commands : free (-m) tells you the total memory, memory used for cache etc.

Python

Issue 1: python ImportError No module named


Resolve 1: The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.

Shell

To remove duplicates in your .bash_history file, export this variable in the .bashrc file:
HISTCONTROL=erasedups

To increase the .bash_history size from 500 to 1000 commands, export this variable in the .bashrc file:
HISTSIZE=1000

alias hs='history | grep $1' 

Detach a screen if its not dead and is attached to some other terminal

screen -x -R

Few interview questions.

Q1) Factorial of a large num ?
The main difficulty in computing factorials is the size of the result.

Constant Limit = 1000;           %Sufficient digits.
Constant Base = 10;              %The base of the simulated arithmetic.
Constant FactorialLimit = 365;   %Target number to solve, 365!
Array digit[1:Limit] of integer; %The big number.
Integer carry,d;                 %Assistants during multiplication.
Integer last,i;                  %Indices to the big number's digits.
Array text[1:Limit] of character;%Scratchpad for the output.
Constant tdigit[0:9] of character = ["0","1","2","3","4","5","6","7","8","9"];
BEGIN
 digit:=0;                       %Clear the whole array.
 digit[1]:=1;                    %The big number starts with 1,
 last:=1;                        %Its highest-order digit is number 1.
 for n:=1 to FactorialLimit do   %Step through producing 1!, 2!, 3!, 4!, etc. 
  carry:=0;                      %Start a multiply by n.
  for i:=1 to last do            %Step along every digit.
   d:=digit[i]*n + carry;        %The classic multiply.
   digit[i]:=d mod Base;         %The low-order digit of the result.
   carry:=d div Base;            %The carry to the next digit.
  next i;
  while carry > 0                %Store the carry in the big number.            
   if last >= Limit then croak('Overflow!'); %If possible!
   last:=last + 1;               %One more digit.
   digit[last]:=carry mod Base;  %Placed.
   carry:=carry div Base;        %The carry reduced.
  Wend                           %With n > base, maybe > 1 digit extra.
  text:=" ";                     %Now prepare the output.
  for i:=1 to last do            %Translate from binary to text.
   text[Limit - i + 1]:=tdigit[digit[i]]; %Reversing the order.
  next i;                        %Arabic numerals put the low order last.
  Print text," = ",n,"!";        %Print the result!
 next n;                         %On to the next factorial up.
END;

database operations

http://www.1keydata.com/sql/sql-create-index.html

 A  table index helps SQL statements run faster. The syntax for creating an index is:

CREATE INDEX IDX_CUSTOMER_LAST_NAME
on CUSTOMER (Last_Name)

 
 Copying table:
CREATE TABLE copyname SELECT * FROM originalname 

Creating key across multiple columns :
alter table tbl_name add primary key(col1,col2,col3);

If there are duplicates rows in a db across this combination:
alter ignore table judgments add unique index  `idx_nm` (col1,col2,col3);
and then drop the idx: alter ignore table judgments drop idx_name



Connections:
Q)How to check the number of connections to a db ?
A) show processlist;
 
Q)How to see the current indexes?
A) show indexes from table_name;

View all files installed by a package with Debian

dpkg -l fortune : matches all packages with name containing fortune
dpkg -l fortunes-min: specifies files installed by package fortunes-min




Interesting commands:
cowsay , fortune ,toilet

Simple use of cut

ls -l | cut -d " " -f 1 : to print the permissions only
ls -l | cut -d " " -f2- : for the rest of fields

Nice post to detect ports.

http://www.redhat.com/docs/manuals/linux/RHL-9-Manual/security-guide/s1-server-ports.html

nmap -sT -O localhost

netstat -anp | grep 834 

lsof -i | grep 834

Scrolling while using screens

I had missed this for long:

If you are using screen and you want to scroll through the terminal use the following sequence:

ctrl+a [

After this you can use the arrow keys or ctrl + u,b etc

http://www.samsarin.com/blog/2007/03/11/gnu-screen-working-with-the-scrollback-buffer/

mysql Dumps!

mysql> SELECT * FROM  INTO 
OUTFILE '' FIELDS TERMINATED BY ',';

We can use below query for importing data from .txt file

mysql> LOAD DATA INFILE  INTO 
TABLE ''
FIELDS TERMINATED BY ',';

Installing CPAN for perl

Installing: install cpan/
Reloading: reload cpan
dh-make-perl --install --cpan Template::Extract  :          for the module Extract from Template

Caching

 Excellent article on caching on http://www.mnot.net/cache_docs/

What’s a Web Cache? Why do people use them?

1. Kinds of Web Caches
   
   1. Browser Caches
   2. Proxy Caches
2. Aren’t Web Caches bad for me? Why should I help them?
3. How Web Caches Work
4. How (and how not) to Control Caches
   
   1. HTML Meta Tags vs. HTTP Headers
   2. Pragma HTTP Headers (and why they don’t work)
   3. Controlling Freshness with the Expires HTTP Header
   4. Cache-Control HTTP Headers
   5. Validators and Validation
5. Tips for Building a Cache-Aware Site
6. Writing Cache-Aware Scripts
7. Frequently Asked Questions
8. Implementation Notes — Web Servers
9. Implementation Notes — Server-Side Scripting
10. References and Further Information
11. About This Document

 

PHP memcache

What is Memcached?(http://memcached.org/)

Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.
Memcached is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches. Its API is available for most popular languages.


Memcache module provides handy procedural and object oriented interface to memcached, highly effective caching daemon, which was especially designed to decrease database load in dynamic web applications.

sudo apt-get install php5-memcache

Example:


$memcache = new Memcache;$memcache->connect('localhost', 11211) or die ("Could not connect");
$version = $memcache->getVersion();
echo "Server's version: ".$version."
\n";
$tmp_object = new stdClass;$tmp_object->str_attr = 'test';$tmp_object->int_attr = 123;
$memcache->set('key', $tmp_object, false, 10) or die ("Failed to save data at the server");
echo "Store data in the cache (data will expire in 10 seconds)
\n";
$get_result = $memcache->get('key');
echo "Data from the cache:
\n";
var_dump($get_result);
?>

Memcache Functions

Table of Contents

• Memcache::add — Add an item to the server
• Memcache::addServer — Add a memcached server to connection pool
• Memcache::close — Close memcached server connection
• Memcache::connect — Open memcached server connection
• memcache_debug — Turn debug output on/off
• Memcache::decrement — Decrement item's value
• Memcache::delete — Delete item from the server
• Memcache::flush — Flush all existing items at the server
• Memcache::get — Retrieve item from the server
• Memcache::getExtendedStats — Get statistics from all servers in pool
• Memcache::getServerStatus — Returns server status
• Memcache::getStats — Get statistics of the server
• Memcache::getVersion — Return version of the server
• Memcache::increment — Increment item's value
• Memcache::pconnect — Open memcached server persistent connection
• Memcache::replace — Replace value of the existing item
• Memcache::set — Store data at the server
• Memcache::setCompressThreshold — Enable automatic compression of large values
• Memcache::setServerParams — Changes server parameters and status at runtime

PECL

What is PECL?

PECL is a repository for PHP Extensions, providing a directory of all known extensions and hosting facilities for downloading and development of PHP extensions.
The packaging and distribution system used by PECL is shared with its sister, PEAR.

To see the list of packages go here:

http://pecl.php.net/packages.php

PHP APC

The Alternative PHP Cache (APC) is a free and open opcode cache for PHP. Its goal is to provide a free, open, and robust framework for caching and optimizing PHP intermediate code.

Check for the installation using:php -r 'phpinfo();' |grep apc 
If its not installed already use(on ubuntu):
  
 sudo apt-get install php-apc
 sudo /etc/init.d/apache2 restart
 or
 apt-get install apache2-prefork-dev
 pecl install apc
 
Use dpkg -L php-apc to know where the files are installed

There are two primary decisions to be made configuring APC. First, how much memory is going to be allocated to APC; and second, whether APC will check if a file has been modified on every request. The two ini directives that control these settings are apc.shm_size and apc.stat, respectively. Read the sections on these two directive carefully below. 



APC Functions

Table of Contents

• apc_add — Cache a variable in the data store
• apc_cache_info — Retrieves cached information from APC's data store
• apc_clear_cache — Clears the APC cache
• apc_compile_file — Stores a file in the bytecode cache, bypassing all filters.
• apc_define_constants — Defines a set of constants for retrieval and mass-definition
• apc_delete — Removes a stored variable from the cache
• apc_fetch — Fetch a stored variable from the cache
• apc_load_constants — Loads a set of constants from the cache
• apc_sma_info — Retrieves APC's Shared Memory Allocation information
• apc_store — Cache a variable in the data store

Eg:

1)$bar = 'BAR';apc_store('foo', $bar);var_dump(apc_fetch('foo'));?>

 

outputs : BAR 


2)

$constants = array(
    'ONE'   => 1,
    'TWO'   => 2,
    'THREE' => 3,
);apc_define_constants('numbers', $constants);
echo ONE, TWO, THREE;?>

 

From :php.net/apc

 

 

useful javascript functions

1)Window.open(url,id,extra_param) where extra_param may be toolbar=no;width,height etc.
2)window.location(url) to go to another url;

Commands to know

Check the current runlevel :
1)who -r  or  runlevel
Change the runlevel :
init n where n=1to 6 0:halt 1:single user 2-5:multiuser 6:reboot
* man init ,update-rc.d
* link http://www.cyberciti.biz/tips/linux-changing-run-levels.html

Remove the job from running automatically in a run-level
* sudo update-rc.d -f cups remove
* sudo update cups defaults
But if the next time package is upgraded , the links(to /etc/init.d/) will again be made.
So better rename the start script to stop script S to K

Also rc.local is the script which is run after all run-level scripts have been run.Can be pretty useful to put startup commands here.



2) Command line short-cut
  Take arguments from previous command : (!*)
  like touch temp;rm !*; removes it

3)finger -l username : to check the details about a user like logged in,idle time,terminal etc .
  .plan in home dir can be used for additional info
  .nofinger can be used to restrict fingering.

4)sudo apt-get install dict-gcide
To install the webster dictionary on ubuntu

5) To find a directory/file
  find / -name 'dirname' -type d

Vim editor.. I love it

Just found a great tip about vim.
If you have to append say "good" at the end of each line just do this:
:% normal A good
here normal is a command which tell Vi to use a normal char (in this case A for append) as part of command.

Javascript code to check attributes of an object

If you have firebug enabled, you can view the attr of an obj using:
console.log(object);

PHP security issues / bad feature #php

Register Globals (register_globals)
In short, register_globals was meant to help rapid application development. Take for example this URL, http://yoursite.tld/index.php?var=1, which includes a query string. The register_globals statement allows us to access the value with $var instead of $_GET['var'] automatically. This might sound useful to you, but unfortunately all variables in the code now have this property, and we can now easily get into PHP applications that do not protect against this unintended consequence.
 The following code snippet is just one common example you will see in PHP scripts:

eg :if( !empty( $_POST['username'] ) && $_POST['username'] == 'test' && !empty( $_POST['password'] ) && $_POST['password'] == "test123" )
{
    $access = true;
}

f the application is running with register_globals ON, a user could just place access=1 into a query string, and would then have access to whatever the script is running.



Disabling with .htaccess

php_flag register_globals 0

Disabling with php.ini

register_globals = Off

 Ref:http://net.tutsplus.com/tutorials/php/5-helpful-tips-for-creating-secure-php-applications/

PHP security issues #php

Use password hashing along with salts to make the task of hackers as difficult as possible.

In PHP you can generate hashes using the md5() and sha1 functions. md5() returns a 128-bit hash (32 hexadecimal characters), whereas sha1() returns a 160-bit hash (40 hexadecimal characters). For example:

$string = 'PHP & Information Security';
printf("Original string: %s\n", $string);
printf("MD5 hash: %s\n", md5($string));
printf("SHA-1 hash: %s\n", sha1($string));

?>

Using salt to make the hash more random:

define('SALT_LENGTH', 9);

function generateHash($plainText, $salt = null)
{
    if ($salt === null)
    {
        $salt = substr(md5(uniqid(rand(), true)), 0, SALT_LENGTH);
    }
    else
    {
        $salt = substr($salt, 0, SALT_LENGTH);
    }

    return $salt . sha1($salt . $plainText);
}

?>

http://phpsec.org/articles/2005/password-hashing.html

Injection Prevention - mysql_real_escape_string() #mysql

What mysql_real_escape_string does is take a string that is going to be used in a MySQL query and return the same string with all SQL Injection attempts safely escaped. Basically, it will replace those troublesome quotes(') a user might enter with a MySQL-safe substitute, an escaped quote \'.

http://www.tizag.com/mysqlTutorial/mysql-php-sql-injection.php

Bash_profile #linux #shell_programming

Starting with something as simple as difference between .bashrc and .bash_profile.
.bash_profile is executed at login.. where .bashrc is executed each time a shell is created...so don't do things like adding to PATH in .bashrc.




**All my blogs are meant to recount the very basics and if possible help others who might come across it. Its my bugzilla where I am going to record all the problems I face and their solutions.**

~~~~“Don't be pushed by your problems. Be led by your dreams"