They are almost identical but with a few exceptions. For example, we say "start the car" but never "begin the car". An example where we use begin but not start would be "this is only the beginning" but never "this is only the starting". Also, I see the word "begin" more in books than in conversation.