For whatever reason, WordPress does not yet natively support using an SVG as a favicon; fortunately, it is a very basic snippet.
<?php
// Inject SVG Favicon into website head
add_action('wp_head', function() {
echo '<link rel="icon" href="full local path to svg">';
});
// Inject SVG Favicon into admin dashboard head.
add_action('admin_head', function() {
echo '<link rel="icon" href="full local path to svg">';
});Just add this to your functions.php or snippet manager.
There is a significant change to the UI in WP 7.1, and the second snippet serves no purpose.