What is the purpose of having dual citizenship?

Simon Baker of “The Mentalist” recently said he is considering applying for US citizenship, but does not intend to give up his Australian citizenship. Why would somebody want to have dual citizenship?