Is this the proper way to implement a multi-output logger?

Too big interface for my taste.
If you want to use this concurrently, must protect the handlers map from concurrent use/modification.

The easiest would be just have a SetHandler, and not expose Handlers directly.