How to Remove Os.chdir Line When Uploading to Github

Python Files and os.path

---

bogotobogo.com site search:

---

Directories

The module called os contains functions to get information on local directories, files, processes, and surround variables.

os.getcwd()

The current working directory is a property that Python holds in retentivity at all times. There is always a electric current working directory, whether we're in the Python Vanquish, running our own Python script from the command line, etc.

>>> import os >>> print(os.getcwd()) C:\Python32 >>> os.chdir('/exam') >>> print(os.getcwd()) C:\examination        

We used the os.getcwd() function to get the electric current working directory. When nosotros run the graphical Python Shell, the current working directory starts as the directory where the Python Shell executable is. On Windows, this depends on where we installed Python; the default directory is c:\Python32. If nosotros run the Python Shell from the control line, the current working directory starts as the directory nosotros were in when we ran python3.

Then, nosotros used the bone.chdir() function to change the current working directory. Annotation that when we called the os.chdir() office, we used a Linux-style pathname (frontwards slashes, no drive letter) fifty-fifty though nosotros're on Windows. This is ane of the places where Python tries to paper over the differences between operating systems.

os.path.join()

os.path contains functions for manipulating filenames and directory names.

>>> import os >>> print(os.path.join('/test/', 'myfile')) /test/myfile >>> print(bone.path.expanduser('~')) C:\Users\M >>> print(os.path.join(os.path.expanduser('~'),'dir', 'subdir', 'k.py')) C:\Users\K\dir\subdir\k.py        

The bone.path.join() function constructs a pathname out of i or more partial pathnames. In this case, it just concatenates strings. Calling the os.path.join() function will add an extra slash to the pathname before joining it to the filename.

The os.path.expanduser() function will expand a pathname that uses ~ to correspond the electric current user'southward home directory. This works on whatsoever platform where users have a home directory, including Linux, Mac OS 10, and Windows. The returned path does non have a trailing slash, only the os.path.join() function doesn't mind.

Combining these techniques, we can easily construct pathnames for directories and files in the user'south dwelling house directory. The os.path.join() function can take whatsoever number of arguments.

Note: we demand to be careful near the string when nosotros use os.path.bring together. If nosotros utilise "/", it tells Python that we're using absolute path, and it overrides the path earlier information technology:

>>> import bone >>> print(os.path.bring together('/test/', '/myfile')) /myfile        

As nosotros tin see the path "/examination/" is gone!

os.path.split()

os.path also contains functions to split full pathnames, directory names, and filenames into their elective parts.

>>> pathname = "/Users/K/dir/subdir/k.py" >>> os.path.split(pathname) ('/Users/K/dir/subdir', 'thou.py') >>> (dirname, filename) = os.path.split(pathname) >>> dirname '/Users/1000/dir/subdir' >>> pathname '/Users/Grand/dir/subdir/grand.py' >>> filename 'chiliad.py' >>> (shortname, extension) = bone.path.splitext(filename) >>> shortname 'k' >>> extension '.py'        

The split up() function splits a full pathname and returns a tuple containing the path and filename. The os.path.separate() function does return multiple values. We assign the render value of the split function into a tuple of two variables. Each variable receives the value of the corresponding element of the returned tuple. The first variable, dirname, receives the value of the starting time element of the tuple returned from the os.path.divide() office, the file path. The 2nd variable, filename, receives the value of the 2nd chemical element of the tuple returned from the bone.path.split() function, the filename.

os.path also contains the os.path.splitext() function, which splits a filename and returns a tuple containing the filename and the file extension. Nosotros used the same technique to assign each of them to separate variables.

glob.glob()

The glob module is another tool in the Python standard library. It's an easy way to get the contents of a directory programmatically, and information technology uses the sort of wildcards that we may already be familiar with from working on the command line.

>>> import glob >>> bone.chdir('/test') >>> import glob >>> glob.glob('subdir/*.py') ['subdir\\tes3.py', 'subdir\\test1.py', 'subdir\\test2.py']        

The glob module takes a wildcard and returns the path of all files and directories matching the wildcard.

File metadata

Every file system stores metadata about each file: creation date, last-modified appointment, file size, and then on. Python provides a unmarried API to access this metadata. We don't demand to open the file and all we need is the filename.

>>> import os >>> print(bone.getcwd()) C:\test >>> os.chdir('subdir') >>> print(os.getcwd()) C:\test\subdir >>> metadata = os.stat('test1.py') >>> metadata.st_mtime 1359868355.9555483 >>> import time >>> time.localtime(metadata.st_mtime) time.struct_time(tm_year=2013, tm_mon=2, tm_mday=2, tm_hour=21,  tm_min=12, tm_sec=35, tm_wday=5, tm_yday=33, tm_isdst=0) >>> metadata.st_size 1844        

Calling the os.stat() function returns an object that contains several different types of metadata about the file. st_mtime is the modification time, just it's in a format that isn't terribly useful. Actually, it's the number of seconds since the Epoch, which is divers as the first second of January 1st, 1970.

The time module is part of the Python standard library. It contains functions to convert between different time representations, format time values into strings, and fiddle with timezones.

The time.localtime() function converts a fourth dimension value from seconds-since-the-Epoch (from the st_mtime property returned from the bone.stat() function) into a more useful structure of twelvemonth, month, solar day, hour, infinitesimal, 2nd, and and then on. This file was last modified on Feb two, 2013, at around 9:12 PM.

The bone.stat() function also returns the size of a file, in the st_size property. The file "test1.py" is 1844 bytes.

os.path.realpath() - Accented pathname

The glob.glob() part returned a list of relative pathnames. If weu want to construct an absolute pathname - i.due east. i that includes all the directory names back to the root directory or drive letter - then we'll demand the os.path.realpath() function.

>>> import bone >>> impress(os.getcwd()) C:\test\subdir >>> print(os.path.realpath('test1.py')) C:\test\subdir\test1.py        

os.path.expandvars() - Env. variable

The expandvars function inserts environment variables into a filename.

>>> import os >>> os.environ['SUBDIR'] = 'subdir' >>> print(os.path.expandvars('/dwelling/users/Thousand/$SUBDIR')) /habitation/users/K/subdir        

Opening Files

To open up a file, we use built-in open() function:

myfile = open('mydir/myfile.txt', 'westward')        

The open() function takes a filename as an statement. Here the filename is mydir/myfile.txt, and the next argument is a processing mode. The mode is ordinarily the cord 'r' to open text input (this is the default mode), 'w' to create and open up open for text output. The string 'a' is to open for appending text to the cease. The mode argument can specify additional options: calculation a 'b' to the mode string allows for binary data, and adding a + opens the file for both input and output.

The table below lists several combination of the processing modes:

Mode	Clarification
r	Opens a file for reading only. The file pointer is placed at the start of the file. This is the default mode.
rb	Opens a file for reading but in binary format. The file pointer is placed at the kickoff of the file. This is the default way.
r+	Opens a file for both reading and writing. The file pointer will be at the commencement of the file.
rb+	Opens a file for both reading and writing in binary format. The file pointer volition be at the start of the file.
w	Opens a file for writing simply. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
wb	Opens a file for writing only in binary format. Overwrites the file if the file exists. If the file does non be, creates a new file for writing.
due west+	Opens a file for both writing and reading. Overwrites the existing file if the file exists. If the file does not be, creates a new file for reading and writing.
a	Opens a file for appending. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
ab	Opens a file for appending in binary format. The file arrow is at the terminate of the file if the file exists. That is, the file is in the append mode. If the file does not be, it creates a new file for writing.
a+	Opens a file for both appending and reading. The file pointer is at the cease of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
ab+	Opens a file for both appending and reading in binary format. The file pointer is at the stop of the file if the file exists. The file opens in the append mode. If the file does not be, it creates a new file for reading and writing.

At that place are things we should know about the filename:

1. It'southward not just the proper name of a file. It's a combination of a directory path and a filename. In Python, whenever we need a filename, we tin include some or all of a directory path as well.


2. The directory path uses a forward slash without mentioning operating organization. Windows uses astern slashes to denote subdirectories, while Linux utilise frontwards slashes. Merely in Python, forrard slashes always work, even on Windows.


3. The directory path does not begin with a slash or a bulldoze alphabetic character, so it is called a relative path.


4. It's a string. All modern operating systems use Unicode to store the names of files and directories. Python 3 fully supports non-ASCII pathnames.

Character Encoding

A string is a sequence of Unicode characters. A file on disk is not a sequence of Unicode characters but rather a sequence of bytes. Then if we read a file from disk, how does Python catechumen that sequence of bytes into a sequence of characters?
Internally, Python decodes the bytes according to a specific character encoding algorithm and returns a sequence of Unicode graphic symbol string.

I have a file ('Lone.txt'):

나 혼자 (Alone) - By Sistar 추억이 이리 많을까 넌 대체 뭐할까 아직 난 이래 혹시 돌아 올까 봐        

Let'due south try to read the file:

>>> file = open('Solitary.txt') >>> str = file.read() Traceback (most recent call last):   File "", line 1, in                          str = file.read()   File "C:\Python32\lib\encodings\cp1252.py", line 23, in decode     return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position six: character maps to                              >>>                                            

What just happened?
Nosotros didn't specify a character encoding, so Python is forced to use the default encoding.
What's the default encoding? If we wait closely at the traceback, we tin can see that it's crashing in cp1252.py, pregnant that Python is using CP-1252 as the default encoding here. (CP-1252 is a common encoding on computers running Microsoft Windows.) The CP-1252 character set doesn't support the characters that are in this file, so the read fails with an UnicodeDecodeError.

Actually, when I brandish the Korean character, I had to put the following lines of html to the header section:

<!--   <meta http-equiv="Content-Type"  content="text/html; charset=ISO-8859-1" /> -->  <meta http-equiv="Content-Type" content="text/html; charset=utf-8"          />        

ASCII and Unicode

At that place are character encodings for each major language in the world. Since each linguistic communication is different, and retentivity and disk infinite have historically been expensive, each graphic symbol encoding is optimized for a particular language. Each encoding using the same numbers (0-255) to represent that linguistic communication'due south characters. For case, the ASCII encoding, which stores English characters every bit numbers ranging from 0 to 127. (65 is majuscule A, 97 is lowercase a). English has a very simple alphabet, so information technology tin can be completely expressed in less than 128 numbers.

Western European languages like French, Castilian, and German have more letters than English. The most common encoding for these languages is CP-1252. The CP-1252 encoding shares characters with ASCII in the 0-127 range, just then extends into the 128-255 range for characters like ñ, ü, etc. It's all the same a single-byte encoding, though; the highest possible number, 255, withal fits in 1 byte.

And so there are languages like Chinese and Korean, which have so many characters that they require multiple-byte graphic symbol sets. That is, each character is represented by a two-byte number (0-65535). But different multi-byte encodings nevertheless share the same problem as different single-byte encodings, namely that they each apply the aforementioned numbers to mean unlike things. It's only that the range of numbers is broader, because at that place are many more than characters to represent.

Unicode is designed to represent every character from every linguistic communication. Unicode represents each letter, character, or ideograph as a 4-byte number. Each number represents a unique character used in at least one of the world'south languages. At that place is exactly 1 number per graphic symbol, and exactly 1 character per number. Every number always ways just one thing; there are no modes to continue runway of. U+0061 is always 'a', even if a linguistic communication doesn't accept an 'a' in information technology.

This appears to exist a swell idea. Ane encoding to rule them all. Multiple languages per document. No more than manner switching to switch between encodings mid-stream. Only Iv bytes for every single character? That is actually wasteful, especially for languages like English language and Castilian, which demand less than i byte (256 numbers) to express every possible graphic symbol.

Unicode - UTF-32

In that location is a Unicode encoding that uses 4 bytes per grapheme. Information technology's called UTF-32, because 32 bits = 4 bytes. UTF-32 is a straightforward encoding; information technology takes each Unicode grapheme (a iv-byte number) and represents the character with that same number. This has some advantages, the virtually important beingness that we can detect the Nth grapheme of a cord in abiding time, because the Nth character starts at the 4xNth byte. It besides has several disadvantages, the almost obvious being that information technology takes 4 freaking bytes to store every freaking character.

Unicode - UTF-16

Fifty-fifty though there are a lot of Unicode characters, it turns out that virtually people will never employ anything beyond the showtime 65535. Thus, there is another Unicode encoding, called UTF-16 (because 16 $.25 = 2 bytes). UTF-16 encodes every character from 0-65535 every bit 2 bytes, then uses some dirty hacks if nosotros actually demand to represent the rarely-used Unicode characters beyond 65535. Near obvious advantage: UTF-16 is twice as space-efficient every bit UTF-32, because every character requires merely two bytes to store instead of four bytes. And nosotros can even so easily find the Nth character of a cord in constant time.

But there are as well non-obvious disadvantages to both UTF-32 and UTF-16. Different reckoner systems shop private bytes in different ways. That ways that the character U+4E2D could be stored in UTF-16 equally either 4E 2D or 2D 4E, depending on whether the system is large-endian or little-endian. (For UTF-32, there are even more possible byte orderings.)

To solve this trouble, the multi-byte Unicode encodings define a Byte Social club Marker, which is a special non-printable character that we can include at the beginning of our document to point what social club our bytes are in. For UTF-xvi, the Byte Order Mark is U+FEFF. If nosotros receive a UTF-sixteen document that starts with the bytes FF Atomic number 26, we know the byte ordering is i way; if it starts with FE FF, we know the byte ordering is reversed.

Still, UTF-16 isn't exactly ideal, especially if nosotros're dealing with a lot of ASCII characters. If nosotros think about it, fifty-fifty a Chinese web page is going to comprise a lot of ASCII characters - all the elements and attributes surrounding the printable Chinese characters. Being able to find the Nth character in constant time is nice, but nosotros can't guarantee that every grapheme is exactly two bytes, so we can't really find the Nth character in constant time unless we maintain a divide alphabetize.

Unicode - UTF-8

UTF-8 is a variable-length encoding system for Unicode. That is, different characters take up a unlike number of bytes. For ASCII characters (A-Z) UTF-8 uses just 1 byte per grapheme. In fact, it uses the exact same bytes; the showtime 128 characters (0-127) in UTF-8 are indistinguishable from ASCII. Extended Latin characters like ñ and ü end up taking two bytes. (The bytes are not just the Unicode code point similar they would be in UTF-16; there is some serious chip-twiddling involved.) Chinese characters similar 王 stop upward taking iii bytes. The rarely-used astral plane characters take four bytes.

Disadvantages: considering each character tin take a different number of bytes, finding the Nth character is an O(N) performance - that is, the longer the string, the longer information technology takes to detect a specific grapheme. Too, there is bit-twiddling involved to encode characters into bytes and decode bytes into characters.

Advantages: super-efficient encoding of common ASCII characters. No worse than UTF-16 for extended Latin characters. Improve than UTF-32 for Chinese characters. Besides in that location are no byte-ordering issues. A document encoded in utf-8 uses the verbal aforementioned stream of bytes on any computer.

File Object

The open() function returns a file object, which has methods and attributes for getting information about and manipulating a stream of characters.

>>> file = open('Lone.txt') >>> file.manner 'r' >>> file.name 'Lone.txt' >>> file.encoding 'cp1252'        

If we specify the encoding:

>>> # -*- coding: utf-eight -*- >>> file = open('Solitary.txt',          encoding='utf-8') >>> file.encoding 'utf-8' >>> str = file.read() >>> str '나 혼자 (Alone) - Past Sistar\n추억이 이리 많을까 넌 대체 뭐할까\n아직 난 이래 혹시 돌아 올까 봐\north'        

The first line was encoding declaration which needed to make the Python aware of Korean.

The name attribute reflects the proper name nosotros passed in to the open up() function when we opened the file. The encoding aspect reflects the encoding we passed in to the open() function. If we didn't specify the encoding when we opened the file, then the encoding attribute will reflect locale.getpreferredencoding(). The mode aspect tells the states in which mode the file was opened. We can laissez passer an optional fashion parameter to the open() office. Nosotros didn't specify a manner when we opened this file, so Python defaults to 'r', which means open up for reading but, in text mode. The file mode serves several purposes; different modes allow united states write to a file, suspend to a file, or open a file in binary mode.

read()

>>> file = open('Alone.txt', encoding='utf-8') >>> str = file.read() >>> str '나 혼자 (Alone) - Past Sistar\n추억이 이리 많을까 넌 대체 뭐할까\n아직 난 이래 혹시 돌아 올까 봐\northward' >>> file.read() ''        

Reading the file again does not raise an exception. Python does non consider reading by end-of-file to be an error; it but returns an empty cord.

>>> file.read() ''        

Since nosotros're nevertheless at the end of the file, further calls to the stream object's read() method simply return an empty string.

>>> file.seek(0) 0        

The seek() method moves to a specific byte position in a file.

>>> file.read(10) '나 혼자 (Alon' >>> file.seek(0) 0 >>> file.read(fifteen) '나 혼자 (Alone) - ' >>> file.read(1) 'B' >>> file.read(10) 'y Sistar\n추' >>> file.tell() 34        

The read() method can accept an optional parameter, the number of characters to read. We can as well read 1 character at a time. The seek() and tell() methods ever count bytes, only since we opened this file every bit text, the read() method counts characters. Korean characters require multiple bytes to encode in UTF-eight. The English language characters in the file merely crave one byte each, so nosotros might be misled into thinking that the seek() and read() methods are counting the same thing. Simply that'southward only true for some characters.

shut()

It'south important to close files as soon equally we're done with them considering open files eat arrangement resources, and depending on the file mode, other programs may not be able to access them.

>>> file.close() >>> file.read() Traceback (most contempo call last):   File "", line one, in                          file.read() ValueError: I/O operation on closed file. >>> file.seek(0) Traceback (well-nigh contempo call last):   File "", line one, in                                  file.seek(0) ValueError: I/O functioning on airtight file. >>> file.tell() Traceback (about recent call final):   File "", line 1, in                                          file.tell() ValueError: I/O functioning on closed file. >>> file.close() >>> file.closed True                                                                                                  

1. We can't read from a closed file; that raises an IOError exception.


2. We tin can't seek in a closed file either.


3. There's no current position in a closed file, so the tell() method likewise fails.


4. Calling the close() method on a stream object whose file has been closed does not raise an exception. It'due south just a no-op.


5. Closed stream objects exercise have 1 useful attribute: the closed aspect volition confirm that the file is closed.

"with" statement

Stream objects have an explicit close() method, but what happens if our code has a problems and crashes earlier we call close()? That file could theoretically stay open for longer than necessary.

Probably, nosotros could use the try..finally block. But we have a cleaner solution, which is now the preferred solution in Python 3: the with argument:

>>>          with          open('Lonely.txt', encoding='utf-8') as file: 	file.seek(16) 	char = file.read(1) 	print(char)  	 16 o        

The code above never calls file.close(). The with statement starts a code cake, similar an if statement or a for loop. Inside this code block, we can utilise the variable file equally the stream object returned from the telephone call to open(). All the regular stream object methods are bachelor - seek(), read(), whatever we need. When the with block ends, Python calls file.close() automatically.

Note that no matter how or when nosotros exit the with block, Python will close that file even if we exit it via an unhandled exception. In other words, even if our code raises an exception and our entire program comes to a halt, that file will go closed. Guaranteed.

Actually, the with argument creates a runtime context. In these examples, the stream object acts equally a context managing director. Python creates the stream object file and tells it that it is inbound a runtime context. When the with code block is completed, Python tells the stream object that it is exiting the runtime context, and the stream object calls its ain close() method.

There's nothing file-specific near the with statement; it's just a generic framework for creating runtime contexts and telling objects that they're entering and exiting a runtime context. If the object in question is a stream object, and so it closes the file automatically. But that behavior is defined in the stream object, non in the with statement. There are lots of other ways to utilize context managers that accept null to exercise with files.

Reading lines ane by i

A line of text is a sequence of characters delimited by what exactly? Well, it's complicated, considering text files can apply several different characters to marking the cease of a line. Every operating system has its ain convention. Some use a wagon return character(\r), others use a line feed character(\due north), and some utilize both characters(\r\n) at the end of every line.

However, Python handles line endings automatically by default. Python will figure out which kind of line catastrophe the text file uses and and it will all the work for u.s.a..

# line.py  lineCount = 0 with open up('Daffodils.txt', encoding='utf-eight') as file:     for line in file:         lineCount += i         print('{:<5} {}'.format(lineCount, line.rstrip()))        

If we run it:

C:\Examination> python line.py 1     I wandered lonely as a cloud ii     That floats on high o'er vales and hills, 3     When all at once I saw a crowd, 4     A host, of gold daffodils;        

1. Using the with design, we safely open the file and let Python close it for us.


2. To read a file one line at a time, use a for loop. That's it. Besides having explicit methods like read(), the stream object is besides an iterator which spits out a unmarried line every time nosotros ask for a value.


3. Using the format() string method, nosotros can print out the line number and the line itself. The format specifier {:<five} ways print this argument left-justified within v spaces. The a_line variable contains the consummate line, carriage returns and all. The rstrip() string method removes the trailing whitespace, including the carriage return characters.

write()

We can write to files in much the same way that we read from them. First, nosotros open a file and get a file object, so we use methods on the stream object to write information to the file, then close the file.

The method write() writes a string to the file. In that location is no return value. Due to buffering, the string may non actually show upwardly in the file until the flush() or close() method is called.

To open a file for writing, utilise the open() function and specify the write mode. In that location are 2 file modes for writing every bit listed in the earlier table:

1. write mode will overwrite the file when the manner='westward' of the open() function.


2. append mode will add information to the end of the file when the way='a' of the open() function.

We should always close a file every bit presently as we're washed writing to it, to release the file handle and ensure that the data is actually written to disk. Every bit with reading information from a file, nosotros tin telephone call the stream object's close() method, or we can utilise the with statement and let Python shut the file for united states.

>>> with open('myfile', mode='w', encoding='utf-viii') as file: 	file.write('Re-create and paste is a design error.')  >>> with open('myfile', encoding='utf-8') equally file: 	print(file.read()) 	 Re-create and paste is a blueprint error. >>>  >>> with open('myfile', style='a', encoding='utf-8') equally file: 	file.write('\nTesting shows the presence, not the absence of bugs.')  >>> with open('myfile', encoding='utf-8') as file: 	print(file.read())  Copy and paste is a design error.	 Testing shows the presence, not the absence of bugs.        

Nosotros startedby creating the new file myfile, and opening the file for writing. The mode='w' parameter means open up the file for writing. We can add data to the newly opened file with the write() method of the file object returned by the open() function. Later the with block ends, Python automatically closes the file.

So, with mode='a' to suspend to the file instead of overwriting it. Appending will never harm the existing contents of the file. Both the original line we wrote and the 2d line we appended are now in the file. Also note that neither carriage returns nor line feeds are included. Note that nosotros wrote a line feed with the '\due north' character.

Binary files

Motion-picture show file is non a text file. Binary files may contain any type of information, encoded in binary class for computer storage and processing purposes.

Binary files are commonly thought of equally being a sequence of bytes, which ways the binary digits (bits) are grouped in eights. Binary files typically comprise bytes that are intended to be interpreted as something other than text characters. Compiled calculator programs are typical examples; indeed, compiled applications (object files) are sometimes referred to, especially by programmers, as binaries. Simply binary files tin also contain images, sounds, compressed versions of other files, etc. - in curt, any blazon of file content whatever.
Some binary files contain headers, blocks of metadata used by a computer plan to interpret the data in the file. For example, a GIF file tin comprise multiple images, and headers are used to identify and draw each block of paradigm data. If a binary file does not comprise any headers, it may be called a flat binary file. But the presence of headers are also common in plain text files, like email and html files. - wiki

>>> my_image = open('python_image.png', mode='rb') >>> my_image.manner 'rb' >>> my_image.proper noun 'python_image.png' >>> my_image.encoding Traceback (nigh recent phone call last):   File "", line 1, in                          my_image.encoding AttributeError: '_io.BufferedReader' object has no attribute 'encoding'                              

Opening a file in binary manner is elementary merely subtle. The only difference from opening it in text mode is that the way parameter contains a 'b' character. The stream object we get from opening a file in binary mode has many of the aforementioned attributes, including fashion, which reflects the mode parameter we passed into the open() function. Binary file objects also have a name aspect, just like text file objects.

However, a binary stream object has no encoding attribute. That'southward because we're reading bytes, not strings, and then there's no conversion for Python to do.

Let'south continue to do more investigation on the binary:

>>> my_image.tell() 0 >>> image_data = my_image.read(5) >>> image_data b'\x89PNG\r' >>> type(image_data)                      >>> my_image.tell() 5 >>> my_image.seek(0) 0 >>> image_data = my_image.read() >>> len(image_data) 14922                  

Similar text files, we tin can read binary files a niggling bit at a time. As mentioned previously, there's a crucial divergence. We're reading bytes, not strings. Since we opened the file in binary style, the read() method takes the number of bytes to read, non the number of characters.

That means that there'southward never an unexpected mismatch between the number we passed into the read() method and the position index we get out of the tell() method. The read() method reads bytes, and the seek() and tell() methods track the number of bytes read.

read()

size parameter

We can read a stream object is with a read() method that takes an optional size parameter. And then, the read() method returns a string of that size. When chosen with no size parameter, the read() method should read everything there and return all the information as a single value. When called with a size parameter, it reads that much from the input source and returns that much data. When called again, it picks up where information technology left off and returns the next chunk of information.

>>> import io >>> my_string = 'C is quirky, flawed, and an enormous success. - Dennis Ritchie (1941-2011)' >>> my_file = io.StringIO(my_string) >>> my_file.read() 'C is quirky, flawed, and an enormous success. - Dennis Ritchie (1941-2011)' >>> my_file.read() '' >>> my_file.seek(0) 0 >>> my_file.read(x) 'C is quirk' >>> my_file.tell() 10 >>> my_file.seek(x) 10 >>> my_file.read() 'y, flawed, and an enormous success. - Dennis Ritchie (1941-2011)'        

The io module defines the StringIO course that nosotros tin use to treat a cord in memory equally a file. To create a stream object out of a cord, create an case of the io.StringIO() class and laissez passer it the string we desire to utilize every bit our file information. Now we have a stream object, and we tin can practice all sorts of stream-similar things with it.

Calling the read() method reads the entire file, which in the example of a StringIO object simply returns the original string.

We can explicitly seek to the beginning of the string, just like seeking through a real file, by using the seek() method of the StringIO object. We can also read the string in chunks, past passing a size parameter to the read() method.

Reading compressed files

The Python standard library contains modules that support reading and writing compressed files. There are a number of different compression schemes. The two most pop on non-Windows systems are gzip and bzip2.

Though it depends on the intended application. gzip is very fast and has small memory footprint. bzip2 tin can't compete with gzip in terms of speed or memory usage. bzip2 has notably meliorate pinch ratio than gzip, which has to be the reason for the popularity of bzip2; it is slower than gzip specially in decompression and uses more memory.

Data from gzip vs bzip2.

The gzip module lets us create a stream object for reading or writing a gzip-compressed file. The stream object it gives us supports the read() method if we opened it for reading or the write() method if we opened it for writing. That means we tin can utilize the methods nosotros've already learned for regular files to directly read or write a gzip-compressed file, without creating a temporary file to shop the decompressed information.

>>> import gzip >>> with gzip.open('myfile.1000', fashion='wb') equally compressed: 	compressed.write('640K ought to exist enough for anybody (1981). - Beak Gates(1981)'.encode('utf-eight'))  $ ls -l myfile.gz -rwx------+ 1 Administrators None 82 Jan  3 22:38 myfile.gz  $ gunzip myfile.gz  $ true cat myfile 640K ought to be enough for anybody (1981). - Neb Gates(1981)        

We should e'er open up gzipped files in binary fashion. (Notation the 'b' character in the mode argument.) The gzip file format includes a stock-still-length header that contains some metadata about the file, and so information technology'due south inefficient for extremely minor files.

The gunzip command decompresses the file and stores the contents in a new file named the same as the compressed file only without the .gz file extension. The cat command displays the contents of a file. This file contains the string nosotros wrote directly to the compressed file myfile.gz from within the Python Shell.

stdout and stderr

stdin, stdout, and stderr are pipes that are congenital into every organisation such as Linux and MacOSX . When we call the impress() function, the thing we're printing is sent to the stdout pipe. When our program crashes and prints out a traceback, it goes to the stderr pipe. By default, both of these pipes are just continued to the terminal. When our program prints something, nosotros encounter the output in our terminal window, and when a program crashes, we see the traceback in our terminal window besides. In the graphical Python Shell, the stdout and stderr pipes default to our IDE Window.

>>> for northward in range(2): 	print('Coffee is to JavaScript what Machine is to Carpet') 	 Coffee is to JavaScript what Car is to Carpet Coffee is to JavaScript what Car is to Carpet  >>> import sys >>> for n in range(2): 	southward = sys.stdout.write('Simplicity is prerequisite for reliability. ') 	 Simplicity is prerequisite for reliability. Simplicity is prerequisite for reliability.  	 >>> for n in range(2): 	s = sys.stderr.write('stderr ') 	 stderr stderr        

The stdout is defined in the sys module, and it is a stream object. Calling its write() part volition print out whatsoever cord nosotros give, and then return the length of the output. In fact, this is what the print() function really does; it adds a carriage render to the cease of the string nosotros're printing, and calls sys.stdout.write.

sys.stdout and sys.stderr send their output to the aforementioned place: the Python ide if nosotros're in , or the terminal if we're running Python from the command line. Like standard output, standard error does not add wagon returns for us. If we want carriage returns, we'll need to write carriage render characters.

Annotation that stdout and stderr are write-only. Attempting to phone call their read() method will always heighten an IOError.

>>> import sys >>> sys.stdout.read() Traceback (most recent call last):   File "", line 1, in                          sys.stdout.read() AttributeError: read                              

stdout redirect

stdout and stderr only back up writing just they're not constants. They're variables! That means nosotros can assign them a new value to redirect their output.

#redirect.py import sys form StdoutRedirect:     def __init__(self, newOut):         self.newOut = newOut      def __enter__(self):         self.oldOut = sys.stdout         sys.stdout = self.newOut      def __exit__(self, *args):         sys.stdout = self.oldOut  impress('X') with open('output', mode='w', encoding='utf-8') as myFile:     with StdoutRedirect(myFile): 	impress('Y') print('Z')        

If nosotros run it:

$ python redirect.py X Z  $ cat output Y        

We actually have two with statements, i nested within the scope of the other. The outer with statement opens a utf-eight-encoded text file named output for writing and assigns the stream object to a variable named myFile.

However,

with StdoutRedirect(myFile):        

Where's the as clause?
The with statement doesn't actually require one. We can take a with statement that doesn't assign the with context to a variable. In this case, we're simply interested in the side effects of the StdoutRedirect context.

What are those side effects?
Take a look inside the StdoutRedirect course. This class is a custom context manager. Whatsoever class tin can be a context manager past defining 2 special methods: __enter__() and __exit__().

The __init__() method is called immediately later an instance is created. Information technology takes ane parameter, the stream object that we want to apply as standard output for the life of the context. This method simply saves the stream object in an instance variable so other methods can apply it subsequently.

The __enter__() method is a special class method. Python calls it when entering a context (i.e. at the beginning of the with argument). This method saves the current value of sys.stdout in self.oldOut, then redirects standard output past assigning self.newOut to sys.stdout.

__exit__() method is another special class method. Python calls it when exiting the context (i.east. at the end of the with statement). This method restores standard output to its original value by assigning the saved self.oldOut value to sys.stdout.

This with statement takes a comma-separated list of contexts. The comma-separated list acts like a series of nested with blocks. The offset context listed is the outer cake; the last one listed is the inner block. The first context opens a file; the 2d context redirects sys.stdout to the stream object that was created in the first context. Because this print() function is executed with the context created by the with argument, information technology volition not print to the screen; it will write to the file output.

Now, the with lawmaking cake is over. Python has told each context director to do whatever information technology is they practise upon exiting a context. The context managers class a final-in-commencement-out stack. Upon exiting, the second context changed sys.stdout back to its original value, then the first context closed the file named output. Since standard output has been restored to its original value, calling the print() role will once once again print to the screen.

File read/write - sample

The following example shows some other example of reading and writing. It reads two data file (linux word dictionary, and acme-level country domain names such every bit .us, .ly etc.), and find the combination of the two for a given length of the full domain proper name.

# Finding a combination of words and domain proper noun (.ly, .united states of america, etc). LENGTH = 8  d_list = [] with open('domain.txt', 'r') every bit df:     for d in df:         d_list.append((d[0:2]).lower()) impress d_list[:10] d_list = ['us','ly']  wf = open('words.txt', 'r')  w_list = wf.read().separate() wf.close()  print len(w_list) print w_list[:10]  with open up('domain_out.txt', 'westward') as outf:     for d in d_list:         print '------- ', d, ' ------\northward'         outf.write('------- ' + d +  ' ------\northward')         for w in w_list:             if w[-2:] == d and len(west) == LENGTH:                 print due west[:-two] + '.' + d                 outf.write(due west[:-two] + '.' + d + '\n')        

Sample output:

------- the states ------ ... enormo.united states of america exiguo.us fabulo.usa genero.us glorio.u.s.a. gorgeo.united states of america ... virtuo.us vitreo.united states of america wondro.us ------- ly ------ Connol.ly Kimber.ly Thessa.ly abject.ly precipitous.ly absent-minded.ly cool.ly active.ly actual.ly ...        

keyword finally

The keyword finally makes a difference if our lawmaking returns early:

effort:     code1() except TypeError:     code2()     return None          finally:     other_code()        

With this code, the finally block is assured to run earlier the method returns. The cases when this could happen:

1. If an exception is thrown within the except block.
2. If an exception is thrown in run_code1() but it's non a TypeError.
3. Other control flow statements such as continue and interruption statements.

However, without the finally block:

endeavor:     run_code1() except TypeError:     run_code2()     render None    other_code()        

the other_code() doesn't get run if at that place's an exception.

swiftburtooked.blogspot.com

Source: https://www.bogotobogo.com/python/python_files.php

Share this post