How to split large file into several smaller files – Linux
Have you ever want to split a large file into several small files? I’ve face this problem few days ago. I need to split a large file (3GB log file) into several smaller file where i can read it using normal text editor.
To split large file into several smaller files, you can use split command in linux. Just follow the steps below and you will be able to split large file into smaller files.
- in your shell key in
$ split –bytes=1m /path/to/large/file
/path/to/output/file/prefix - Done. You just split your large file into several smaller files
* You can change the output file size by changing the –bytes=1m to your preference. You can use b, k, or m. b represent bytes, k represent kilobytes, m represent megabytes.
To restore the original file, you can use cat command.
To join all the smaller file to restore the original file type:-
$ cat prefix* > NEWFILENAME
Tags: file splitter, linux file splitter, linux split file, split large file, split large file linux, split large file to smaller file
Posted at March 1st, 2007 by chua
If you think this article helps you to solve your problem and clear your headache, feel free to buy me a drink :)
April 26th, 2007 at 3:55 am
thanks for this post! i found it ’cause i needed it.
one thing, there should be two dashes before ‘bytes’.
split –bytes=2m /large/file /smaller/files/prefix
May 4th, 2007 at 2:32 pm
Just in case anybody needed (I did): if you use this command to split a binary file on *NIX and then copy the output to a DOS-aware system (Windows, for that matter) you can concatenate all the chunks using:
$> copy /B chunk* output
September 9th, 2007 at 11:46 pm
Very nice, but I need to recover the original file in a windows XP system. Command copy not found in execute prompt. What can I do then? Thx.
September 28th, 2007 at 11:32 pm
If winxp does not have the copy command you can download the linux cat command for windows. Go to http://unxutils.sourceforge.net/ and download the unix utils, they’re free.
Then you can restore the files just as with the linux example above.
March 25th, 2008 at 4:44 pm
[...] check this link. [...]
August 26th, 2008 at 3:48 am
I’m using cygwin on WXP to try to put my split files back together, but I cannot cat a file larger than 4GB. Is there another way?
August 27th, 2008 at 4:28 am
actualy in windows you can do:
$> type filepart1 filepart2 > filefull
August 27th, 2008 at 4:32 am
ah forgot to say thanks i needed that to split a huge file i needed to give it to a friend on 2 512Mb USB-Flashs
(using linux makes u smarter )
the command i figured it out because it’s dos equivalent to cat
it’s better like that then to use zip multipart, it’s make u cooler in your friends eyes
September 15th, 2008 at 1:54 pm
[...] split -b200m book.rar book-split Here 200m is 200 MegaByte book.rar is the source file to be splitted. and book-split is prefix of generated file. To restore the file: cat book-split* > NEWFILENAME Source: http://www.computerhope.com/unix/usplit.htm http://www.techiecorner.com/107/how-to-split-large-file-into-several-smaller-files-linux/ [...]
October 28th, 2008 at 7:06 am
[...] ref: http://www.techiecorner.com/107/how-to-split-large-file-into-several-smaller-files-linux/ [...]
February 13th, 2009 at 11:34 pm
can’t use “type” on windows if you’re working with binary files. great for ascii though. type translates some control characters so output is not the same as input for binary files, which therefore end up corrupted.
July 25th, 2009 at 3:08 am
While you can’t use the Type command, you can use copy /b on windows:
copy /b “file.01″+”file.02″+”file.03″ “file”
The /b switches the command into binary mode.
August 21st, 2009 at 5:44 pm
What would the command be to split a 2.52gig file in half in Linux?
October 18th, 2009 at 2:10 am
Thanks! I had to import a ~50 MB sql file on a remote server. It would take too long for the sql to script to run and throw security errors. It was difficult to work with a ~50MB sql file as most text editors would lock up. This command, with some slight alteration, worked great as I was able to split the file up into smaller chunks and import chunk by chunk… thanks for the tip!
November 19th, 2009 at 1:44 pm
How should I split a ‘.3gp’ file so that I can upload to facebook without exceeding their upload size limit?
I tried using split –bytes1024K V181109_13.50.3gp pref
.. but this produced two files both of which Quicktime can not play.
I renamed the files with a 3gp suffix before I tried playing them but that, I think is not relevent.
Jim
November 19th, 2009 at 1:45 pm
Oh and one more thing is I’m using cygwin on winXP to perform the split.
Jim
November 24th, 2009 at 8:45 am
hi jim, once u split the file, you cant play it in any player.
You should use Video editor to cut the file and re-render it.
This linux command is not suitable for ur case.
November 24th, 2009 at 2:46 pm
Okay, thanks for your reply.
Do you have any suggestions for a suitable video editor?
Jim
November 24th, 2009 at 3:46 pm
not really familiar with 3gp file.
but i think u can try Ulead media studio, or Adobe Premier.
Both are good for video editing.
December 4th, 2009 at 2:36 am
for 3gp files they have a header that quicktime or other players look for to be able to play it. So you cannot use split to split it to 2 3gp files ever you change the extension. You have to use a special splitter for media files that copy also the header of the file.
December 18th, 2009 at 6:11 am
Hi all,
resurrecting this old interesting post to ask a question about tar.
I’m creating an archive doing:
tar cfz /media/disk/0912171726.tar.gz /data
Is there a way to use ’split’ and a pipe to create a few files WITHOUT creating the big file first?
December 18th, 2009 at 6:15 am
Found it:
http://www.linuxquestions.org/questions/linux-software-2/tar-split-question-605013/
December 28th, 2009 at 2:36 am
Attention to: Grahack
gzip:
tar cfz – |split -b 100M split_file.tgz. (note the last dot, optional though)
gzip with level 9 compression:
tar cf – |gzip -9 -c |split -b 100M split_file.tgz.
bzip2:
tar jcf |split -b 100M split_file.tbz.
bzip2 with level 9 compression:
tar cf – |bzip2 -9 -c |split -b 100M split_file.tbz.
then simply cat split_file* > file.tgz or .tbz and untar/gunzip or bunzip2 accordingly.
In most Linux distro’s, this should work as is. In OpenBSD or FreeBSD, I think the syntax may be a little different – the man pages will say though. As for other *nix OS’s, YMMV. =)
Hope this helps…
December 28th, 2009 at 2:56 am
Whoops, guess I should have proof read that.
After the tar commands, and parameters (jcf, cf, cfz, etc.) and after the -, you’ll of course need to tell it what file(s) to use and/or what directory and after the split command and parameters, just before the output filename, be sure to use the final – as well.
Here’s a better, yet simple example (remove quotes):
“tar cvfp – filename.iso |bzip2 -9 -c |split -a 3 -db 500M – split_file.tbz.”
January 25th, 2010 at 7:00 pm
hello
What does that ‘-’ single character mean in command line?
regards
January 29th, 2010 at 9:25 pm
the – is the way to include additional options.
it’s very common in linux and unix
February 26th, 2010 at 6:50 pm
I think froff meant the really single – .
From the split manpage:
“With no INPUT, or when INPUT is -, read standard input.”
For example, it tells split to read from the pipe (created by the ‘|’) instead of a specified file. Or it tells tar to write to standard output.
By the way, this article / discussion is really helpful!