Geotagging in Ubuntu: more broken than ever

This is depressing. All the geotagging tools in Ubuntu seem dead. Even mine, apparently the latest version of CEF Python changed something that breaks my app. Even worse, looking for “ubuntu geotagging” has my own blog as one of the top results… time to fix my code, I guess?

Advertisements

Fixing templates with constexpr’s

For my hundredth (and a bit) c++ post I decided to do something I never did before: fix my old code!

A long time ago I wrote about template metaprogramming devices. There, I tried to explain that many atrocities have been commited in the name of performance and “compile time evaluation”. Template metaprogramming is probably one of the worse culprits of job security. Its (ab)use can create monstrosities, all in the name of runtime performance. Like, for example, my template device to calculate e. Let’s remember what that atrocious code looks like (follow the link if you want an explanation on how this works):

template <int N, int D> struct Frak {
        static const long Num = N;
        static const long Den = D;
};
  
template <int N, typename F> struct ScalarMultiplication {
    typedef Frak<N*F::Num, N*F::Den> result;
};
  
template <int X, int Y>   struct MCD {
        static const long result = MCD<Y, X % Y>::result;
};
  
template <int X> struct MCD<X, 0> {
        static const long result = X;
};
  
template <class F> struct Simpl {
        static const long mcd = MCD<F::Num, F::Den>::result;
        typedef Frak< F::Num / mcd, F::Den / mcd > result;
};
  
template <typename X1, typename Y1> struct SameBase {
    typedef typename ScalarMultiplication< Y1::Den, X1>::result X;
    typedef typename ScalarMultiplication< X1::Den, Y1>::result Y;
};
  
template <typename X, typename Y> struct Sum {
    typedef SameBase<X, Y> B;
    static const long Num = B::X::Num + B::Y::Num;
    static const long Den = B::Y::Den; // == B::X::Den
    typedef typename Simpl< Frak<Num, Den> >::result result;
};

template <int N> struct Fact {
    static const long result = N * Fact<N-1>::result;
};
template <> struct Fact<0> {
    static const long result = 1;
};
 
template <int N> struct E {
    // e = S(1/n!) = 1/0! + 1/1! + 1/2! + ...
    static const long Den = Fact<N>::result;
    typedef Frak< 1, Den > term;
    typedef typename E<N-1>::result next_term;
    typedef typename Sum< term, next_term >::result result;
};
template <> struct E<0> {
    typedef Frak<1, 1> result;
};
 
int main() {
  typedef E<8>::result X;
  std::cout << "e = " << (1.0 * X::Num / X::Den) << "\n";
  std::cout << "e = " << X::Num <<"/"<< X::Den << "\n";
  return 0;
}

While this is just a toy example to play with templates, it does illustrate code I’ve seen in the wild. Would this look cleaner in c++11? Yes, it would. Constexprs are, in my opinion, one of the most overlooked “killer” features of c++11.

Starting with a simple example:

constexpr int foo(int a, int b) { return a+b; }
static constexpr int n = foo(1, 2);
int bar() { return n; }

Try to compile it with “g++ -std=c++11 -fverbose-asm -O0 -c -S -o /dev/stdout” and see what happens. You should get the equivalent of “return 3” – just as anyone would expect – but note that no optimizations were enabled. What about loops? Let’s try this:

constexpr int f(int n) {
    return (n<2)? 1 : n + f(n-1);
}

constexpr int n = f(999);

You’ll probably get an error about maximum depth exceeded, but that’s alright: we have loops in constexprs too! (note that some of these restrictions have been relaxed in C++17).

In general, if you can express your function as a single const return statement, it should be a valid constexpr. With this new knowledge, let’s convert the template meta-atrocity above to something a bit less hideous:

struct PodFrac {
    int num;
    int den;
};

constexpr int mcd(int a, int b) {
    return (b==0)? a : mcd(b, a%b);
}

constexpr PodFrac simpl(const PodFrac &f) {
    return PodFrac{f.num / mcd(f.num, f.den), f.den / mcd(f.num, f.den)};
}

constexpr PodFrac sum(const PodFrac &a, const PodFrac &b) {
    return simpl(PodFrac{a.num*b.den + b.num*a.den, a.den*b.den});
}

constexpr int fact(int n) {
    return (n==0)? 1 : n*fact(n-1);
}

constexpr PodFrac e(int n) {
    return (n==0)? PodFrac{1, 1} :
                   sum(PodFrac{1, fact(n)}, e(n-1));
}

constexpr float e_num = 1.0 * e(8).num / e(8).den;

float get_e() {
    return e_num;
}

Disclaimer: while I explicitly stated this multiple times in my “C++ template metaprogramming introduction” article, it’s worth re-stating it: this code is meant as an example to showcase a c++ feature, not as a proper way of deriving a mathematical constant in production code.

First thoughts after comparing the two versions: much, much [, much]*100 cleaner.

As you may notice, all constexprs need to be a return statement. There’s no multi-statement constexpr in c++11, which explains why loops are not really supported. For the same reason the implementation of e() is a bit hindered by this limitation: its code would be much more readable splitting it in a few lines with proper names. Good news: some of these restrictions have been relaxed in C++17.

Note that if you analyze your compiler’s output when building without optimizations, you may see either a const with e’s value, or a static initializer that does some trivial operation, like loading e’s value from a fraction: gcc seems to get tired of constexpr evaluation after a few recursive calls, so your results may vary (slightly).

I called constexpr’s one of c++11’s killer features, and hopefully you can see why I’m so enthusiastic about them now: there’s much less incentive for people to write horrible template metaprogramming devices when simply adding a little keyword to a normal function has the same effect, only cleaner.


Quick refresher: argument dependent lookup

Since I wasted a few precious minutes stuck on an ADL problem, I figured I needed a quick reminder on how they work. Check this code: does it compile?

namespace N {
    int foo() {
    }
}

int main() {
    return foo();
}

Of course it doesn’t! You’d expect a ‘foo’ not declared/out of scope error from your compiler. What about this other example?

namespace N {
    struct Dummy;

    int foo(Dummy*) {
        return 0;
    }

    int foo() {
    }
}

int main() {
    return foo((N::Dummy*)0);
}

You’d be tempted to say it won’t work either. (Un?)fortunately, ‘argument dependant lookup’ is a thing, and the second code sample works. How? The compiler will look for ‘foo’ in the global namespace, and also in the namespace of the arguments to ‘foo’. Seeing ‘N::Dummy’ in there, the compiler is allowed to peak into the namespace N for method ‘foo’. Why? Short: operator overloading. Long: check here (the ‘Why ADL’ section is very good).


Google Test: Quarantine for tests?

Google Test: Putting a test under quarantine

GTest works wonders for c++ testing, even more so when combined with GMock. I’ve been using these frameworks for a few side projects. I’ve seen them used in large scale projects too. In all cases, there is a very common problem for which (I think) there is no elegant solution: managing temporarily disabled tests.

It may be because you found a flaky piece of code or a test that exposes a heisenbug. Maybe the test itself is just unstable, or perhaps you are using TDD and want to submit a test to your CI before its implementation is ready. In these cases, you can choose to disable the offending test or let it run, possible halting your CI because of it. When that happens, you maybe masking other, real, problems.

Most people would stick a “DISABLED_” before the test name, to let GTest know not to run it. Maybe even stick a “// TODO: reenable” in there too. When run, GTest will generate a message to let you know there is a disabled test. Even so, I find that people -myself included- tend to forget to re-enable the disabled tests.

For one of my side projects, I hacked GTest to quarantine tests up to a certain date:

TEST(Foo, Bar) {
    QUARANTINE_UNTIL("16/8/22");
    EXPECT_EQ(1,2);
}

In my CI setup, that test will be showing a happy green (and a warning, which I will probably ignore) until the 22nd of August. By the 23rd the test will run again and fail if I haven’t fixed the code. If I have indeed fixed it, it will print a warning to remind me that it’s safe to delete the quarantine statement.

Is there any advantage in this approach over the usual _DISABLE strategy? In my opinion, there is: if you ignore warnings in your test, for whatever reason, a _DISABLE might go unnoticed and it may hide a real problem. In the same scenario, for a quarantined test, nothing bad happens: the warning just says “you should delete this line” but the quarantined test is again part of your safety net.

How does it work? The first caveat in my article mentions it: hackishly. There are a few facilities missing in GTest to make this implementation production-ready but, ugly as it looks, it should work as intended:

#include <ctime>
#include <string>
#include <sstream>
std::string now() {
    time_t t = time(0);
    struct tm *now = localtime(&t);

    std::stringstream formatted_date;
    formatted_date << (now->tm_year+1900) << '/'
                   << (now->tm_mon+1) << '/'
                   << now->tm_mday;

    return formatted_date.str();
}

#define QUARANTINE_UNTIL(date_limit)                                     \
        if (now() < date_limit) {                                        \
            GTEST_LOG_(WARNING) << "Test under quarantine!";             \
            return;                                                      \
        } else {                                                         \
            GTEST_LOG_(WARNING) << "Quarantine expired on " date_limit;  \
        }

If I find there is interest in this approach for real world applications, I may try to come up with a nicer interface for it.


Vimcrypt

Have you ever been working on your plans for world domination but got scared someone else might find them? It happens to me all the time. Or maybe you are so paranoid that you need to encrypt your grocery list. Perhaps you are sharing a semi-private text file through a public service? Good news, Vim has you covered. Just type “:X”. Vim will ask you for a password. Save your file again and voila, your file is now encrypted. Open the same file with Vim to decrypt it. Your plans for world domination are now safe!


Things you should never do

I think I may start a new series in my “Rants” category: things you shouldn’t do. First one: Never ever use “strange” characters on your wifi’s AP name, where strange is defined as non-ascii. I made the huge mistake of choosing a name with an ñ on it, then had to spend an entire evening hacking on a printer driver with no unicode support. No, I couldn’t have changed the AP’s name. That would have required me to physically connect a computer to my router, and I was too lazy to get up from the couch.


Self reminder: setting the default boot option in UEFI

Bought a new laptop (*) and I’m 100% sure I’ll forget this if I don’t put it here:

From http://askubuntu.com/questions/291905/how-can-i-make-ubuntu-the-default-boot-option-on-a-newer-laptop-uefi

To set Ubuntu as the default boot OS in a multi-OS setup (ie, dual boot Windows) with UEFI, goto Windows and exec (as admin) bcdedit /set {bootmgr} path \EFI\ubuntu\grubx64.efi

Why am I using Windows, you may ask? I’m still in the process of discovering which features will be broken and which hardware will work out of the box. So far I’m actually quite surprised, with only the video card and the touchpad not working. Luckily bash doesn’t use either of those. Who needs a mouse anyway?


Simple vim plugin IV: project greping

I recently wrote about some of the utilities I created for my Vim setup. Using someone else’s Vim scripts is not nearly as fun as writing your own, so I decided to also write a short summary on what it takes to get started writting Vim plugins. For this task, I decided to start with greping.

Greping can be improved a bit: if you do it a lot in a project, you might find it’s useful to also grep the results themselves, to further refine your search. If you have your grep results in vim itself, that is trivial.

Let’s start hacking something in our .vimrc file. Try this:

function! FG_Search()
    let needle = input("Search for: ")
    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let grepbin = 'grep -nri '
    let cmd = grepbin . ' "' . needle . '" *'
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

map <leader>s :call FG_Search()<CR>

This function should be pretty clear: it will map <leader>s (in my case, “,s”) to FG_Search(). FG_Search will prompt the user for a term to grep, then search for it executing the command. In the end the results are written to a new tab, which is declared as a temp non-modifiable buffer.

Just paste that in your .vimrc and you’re good to grep.

Extra tip: integrate this with my fast grep cache and you have a nice and quick project search integration for vim that works even for very large projects with tools available in most default Linux installs.


Simple vim plugin III: a polymorphic project greper

We’ve recently seen a very basic function to integrate grep to vim. We can improve it a little bit with very simple changes. Using this tip to have different key binding for different modes we can do something a bit smarter . Let’s create two functions, one for normal mode that should prompt the user what to search for, and another function to automagically pick whatever is selected:

function! FG_DoSearch(needle)
    let grepbin = 'grep -nri '

    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let cmd = grepbin . ' "' . a:needle . '" *'
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

function! FG_Search()
    let needle = input("Search for: ")
    call FG_DoSearch(needle)
endfunction

function! FG_Visual_Search()
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    call FG_DoSearch(needle)
endfunction

nmap <leader>s :call FG_Search()<CR>
vmap <leader>s :call FG_Visual_Search()<CR>

The magic here happens in the mapping: nmap will create a mapping that’s only enabled when on “normal” mode, vmap when you’re in visual mode. As usual, check :help map for more details.


Simple vim plugin II: a psychic project greper

We have been working on a quick grep integration for Vim, and it’s looking decent enough for a quick plugin. There’s one more easy thing we can improve, though: let’s make it psychic! So far we had to tell grep what to look for, either by selecting the text in visual mode or by actually typing the search terms. Typing! That’s so old fashioned. Let’s make grep guess what to look for.

In vim you have a psychic function, expand(“<cword>”). If you call expand(“<cword>”), it will return whatever word is under the cursor. No need to visually select it. If you’re still using the same vimrc definitions, you can do something like

nmap <leader>s :call FG_DoSearch(expand("<cword>"))<CR>

Let’s clean things up a little bit:

" Wrap a grep command: search for needle, show results in a new window
function! FG_DoSearch(needle)
    let grepbin = 'grep -nri '

    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let cmd = grepbin . ' "' . a:needle . '" *'
    echom cmd
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

" Wrap a normal action: ask the user for input, then call func with it
function! FG_RequestInputAction(msg, func)
    let needle = input(a:msg)
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a visual action: call func with whatever is selected under the cursor
function! FG_VAction(func)
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    " Remove whitespaces
    let needle = substitute(needle, "\\n\\+","","g") 
    let needle = substitute(needle, "\\r\\+","","g") 
    let needle = substitute(needle, "^\\s\\+\\|\\s\\+$","","g") 

    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a normal action: call func with whatever is under the cursor
function! FG_NAction(func)
    let needle = expand("<cword>")
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

nmap <leader>s :call FG_NAction("FG_DoSearchText")<CR>
vmap <leader>s :call FG_VAction("FG_DoSearchText")<CR>
map  <leader>S :call FG_RequestInputAction("Text search: ", "FG_DoSearchText")<CR>

Just copy paste that in your vimrc, now you can grep your project in three different ways: press s (,s) to look for the word currently under the cursor, S to type in a search term or select something in visual mode, then S to grep it.