By far, the fastest to read an entire binary file (that I have tested) is: import numpy as np file = "binary_file.bin" data = np.fromfile(file, 'u1') Reference. Multitudes faster than any other methods so far. Hope it helps someone!

for rec in inh: reads one line at a time -- not what you want for a binary file. Read 4 bytes at a time (with a while loop and inh.read(4)) instead (or read everything into memory with a single .read() call, then unpack successive 4-byte slices). The second approach is simplest and most practical as long as the amount of data involved isn't huge:

When you read from a binary file, a data type called bytes is used. This is a bit like list or tuple, except it can only store integers from 0 to 255. Try: file_size = fin.read(4) file_size0 = file_size[0] file_size1 = file_size[1] file_size2 = file_size[2] file_size3 = file_size[3] Or: file_size = list(fin.read(4)) Instead of:

Nov 2, 2016 · This appears at the top of a Google search for reading bits using python. I found bitstring to be a good package for reading bits and also an improvement over the native capability (which isn't bad for Python 3.6) e.g.

Actually it looks like you're trying to read a list (or array) of structures from the file. The idiomatic way to do this in Python is use the struct module and call struct.unpack() in a loop—either a fixed number of times if you know the number of them in advance, or until end-of-file is reached—and store the results in a list.

Sep 6, 2018 · Binary files still support line-by-line reading, where file.readline() will give you the binary data up to the next \n byte. Just open the file as binary, and read one line. Valid UTF-16 data will always have an even length. But because UTF-16 comes in two flavours, big endian and little endian byte orders, you'll need to check how much data ...

May 16, 2013 · When reading binary data with Python I have found numpy.fromfile or numpy.fromstring to be much faster than using the Python struct module. Binary data with mixed types can be efficiently read into a numpy array, using the methods above, as long as the data format is constant and can be described with a numpy data type object (numpy.dtype).

May 25, 2010 · I need to handle tens of Gigabytes data in one binary file. Each record in the data file is variable length. So the file is like: <len1><data1><len2><data2>.....<lenN><dataN> The data contains integer, pointer, double value and so on. I found python can not even handle this situation. There is no problem if I read the whole file in memory.

Apr 11, 2014 · I need to read a binary file, which is composed by several blocks of bytes. For example the header is composed by 6 bytes and I would like to extract those 6 bytes and transform ins sequence of binary characters like 000100110 011001 for example.

Sep 29, 2016 · If you want to make an image out of a binary file, you need to read it in as integer, not float. Currently, the most common format for images is unsigned 8-bit integers. As an example, let's make an image out of the first 10,000 bytes of /bin/bash: