Tried clearing dalvik cache as well, rebooted a few times - changed this option a few times, played with other options as well but could not make it work.
Bottom line is - during ringing it works, after the ringing finishes phone flashes which diminishes the DND goal. Also in the call log a guard icon appears next to the contact.