Coming up Short
Today I want to talk about two functions that are suprisingly complicated, and
seriously misnamed. Those functions are the humble
read and
write functions.
At first glance,
read and
write seem to be incredibly simple.
read takes data from the disk and puts it in a buffer.
write
does exactly the opposite. However, if you look closer, it's a lot more
complicated than that.
Return codes
The first thing to realize is that
read and
write may handle
fewer bytes than requested. So really, they should have been called
try_read and
try_write, or similar. Unfortunately, these "short
reads" and "short writes" happen only very rarely, which means that this
mistake is unlikely to show up in testing. It's a beginner's trap!
It's important to remember than a return code of 0 from
read means that
end-of-file has been reached. On the other hand, a return code of 0 from
write is just another short write, and should be handled like all short
writes.
Signals
Then there's the behavior in the presence of
signals. If a thread is in
the middle of a
read or a
write when a signal arrives, the
operation may return with an error code of -1 and an errno of
EINTR,
meaning no bytes were read or written. It also may return one of the infamous
short reads or writes.
In a sense,
EINTR is just a way for
read to say "I read 0 bytes."
Remember,
read can't return 0 in this scenario because that means
end-of-file. I think there's been at least one case in the Linux kernel of
read returning
EINTR when no signal had, in fact, been delivered.
I guess the idea behind
EINTR is that you can respond more quickly to
signals if they cause your
read or
write operation to return a
little early. The problem with that philosophy is that it assumes that you are
using signals as a form of inter-process communication, which would be a really
bad idea. Someday, I'll probably write an entire blog post about exactly why
that's such a bad idea.
Dealing with It
So how do you deal with this insanity? The simplest way is to not deal with it
at all, and use
fwrite,
fread, and friends.
These functions will not return short reads or writes, or
EINTR.
As another bonus, you get caching in userspace. This means that you
will be making fewer system calls to the kernel, which generally
improves performance.
If someone gives you a file descriptor, you can still use the higher-level
functions. Just call
fdopen to
get a
FILE* associated with that file descriptor. Then you can use
fread and
fwrite on it. But be careful not to mix calls to the
high-level and low-level functions. If you do, you will get inconsistent
results because the high-level functions are buffered and the low-level ones
are not.
If you do need to call the "raw"
read and
write system calls,
you'll need to write a wrapper around them that handles short reads and writes,
as well as
EINTR. This is surprisingly difficult to get right, but very
important to do if you're going to use this interface.
Java Edition
You might think that you could leave all this insanity behind when using a
different programming language. Well, at least in the case of Java, you can't.
InputStream#read
can perform partial reads, the same as the old-fashioned
read system
call.
OutputStream#write
doesn't seem to have the same problem; however, some other write APIs,
like
WritableByteChannel, do.
APIs that return short reads can be more efficient. It's great that we have
these APIs. However, if you're going to create an API like that,
don't call
it read! Call it something like
readUpTo or
tryRead. That
way, at least people will know what they're getting into.
The worst API of all, the all-time stupidest, has to be
InputStream#skip.
The skip operation moves forward "up to N bytes" in an InputStream. These guys
really found the worst features of the POSIX API and blindly copied them.
Short reads are at least somewhat useful because it may be more efficient to
read data in smaller chunks. You may also get improved interactive performance
by interleaving I/O and processing. Of what use are short
skips? Since
you don't get back the data (it was skipped), there's nothing you can do but
call
skip again until it does what you want.
When I asked a few Java programmers what
skip did, none of them knew
about short skips. And it's no surprise: the API is stupid. There's no
conceivable reason you would want a short skip, and nothing in the function
name to suggest that that it could happen to you. And since short skips only
happen every now and then, debugging them is difficult.
Conclusion
Reading and writing may seem like the simplest of operations. But sometimes
appearances can be deceiving. Keep on your toes, read the documentation
carefully, and don't come up short.