War stories: When the visual debugger fails you

I recently had a very strange crash and after some digging I found the lines I suspected the bug to lurk around. They looked something like this:

    const std::string contents = readFile("myFile.txt");
    const std::vector<std::string> lines = utils::split(contents, "\n");
    for (std::string line : lines) {
        if (line.empty()) {
            continue;
        }
        //...
        // Do something elaborate with the line, e.g. printing to console
        std::cout << "<line>" << line.c_str() << "</line>" << std::endl;
    }

The crash occurred in the //... lines because the line was not empty. Wait – what? I tested for emptiness before!
Opening the debugger revealed the following strange situation:

and the above small sample file prints on my (Windows) console:

<line>First line</line>
<line>third line (second one is empty)</line>
<line>fourth line</line>
<line></line>

Scrolling trough the commit history, the problem turned out to be introduced with this change:
OLD CODE (working):

std::string readFile(const std::string& fname, bool binaryMode = false)
{
    std::ios_base::openmode mode = std::ios_base::in;
    if (binaryMode)
        mode |= std::ios_base::binary;

    std::ifstream ifs(fname, mode);
    if (!ifs) {
        throw std::exception(("Couldn't open file: " + fname).c_str());
    }
    return std::string(std::istreambuf_iterator<char>(ifs), std::istreambuf_iterator<char>());
}

NEW CODE (not working):

std::string readFile(const std::string& fname, bool binaryMode = false)
{
    std::ios_base::openmode mode = std::ios_base::in;
    if (binaryMode) {
        mode |= std::ios_base::binary;
    }

    std::ifstream in(fname, mode);
    if (!in) {
        throw std::exception(("Couldn't open file: " + fname).c_str());
    }

    // Used C++ style reading which is more efficient than using stream buffer iterators
    // http://insanecoding.blogspot.de/2011/11/how-to-read-in-file-in-c.html
    std::string contents;
    in.seekg(0, std::ios::end);
    contents.resize(in.tellg());
    in.seekg(0, std::ios::beg);
    in.read(&contents[0], contents.size());
    in.close();

    return contents;
}

The problem was that reading the file with the more efficient solution resulted in the string having a bunch of null terminators if the file contained a new-line in the end. Obviously, .empty() returns false, so the check passed. As a side note: to simulate my crash bug by showing an unexpected console output, I had to pipe the C-string. When piping the C++ string, the line is printed with some whitespaces.

This is how I fixed it:

std::string readFile(const std::string& fname, bool binaryMode = false)
{
    std::ios_base::openmode mode = std::ios_base::in;
    if (binaryMode) {
        mode |= std::ios_base::binary;
    }

    std::ifstream in(fname, mode);
    if (!in) {
        throw std::exception(("Couldn't open file: " + fname).c_str());
    }

    // Used C++ style reading which is more efficient than using stream buffer iterators
    // http://insanecoding.blogspot.de/2011/11/how-to-read-in-file-in-c.html
    std::string contents;
    in.seekg(0, std::ios::end);
    contents.resize(in.tellg());
    in.seekg(0, std::ios::beg);
    in.read(&contents[0], contents.size());
    in.close();

    if (binaryMode) {
        return contents;
    }
    else {
        // Depending on the file, the last line might contain one or more \0 control characters. Remove them
        return contents.erase(contents.find_last_not_of('\0') + 1);
    }
}

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.