Good question! You will see several different reasons touted for why it works, from an evolutionary perspective to a scientific one and lots of health reasons in between.
The big question mark out there for most of us though is whether or not it's been "proven". As we've seen over the last couple decades, proving a scientific theory is not as easy as it sounds (global warming anyone?) Going from theory to fact is such a high bar, that very few new theories reach it within a lifetime, much less a decade or two. If you have that amount of time to wait and aren't looking for a way to be healthier now, then wait and we'll probably understand a lot more about diet in a few decades.
But if you are willing to lower the bar there are many good scientific studies which support the elimination of grains and dairy to reduce the inflammatory responses in our bodies. Ultimately, though, you may just have to try it for yourself like we did. But if you are also struggling with immune system issues, there may be even more foods which are affecting you and seeking out a doctor trained in functional medicine to help you identify them may be necessary. If you just want to feel better, lose some weight, avoid further health issues as you get older, then eating this way should be very helpful.
A lot of people find that reading either the Paleo Diet or Paleo Solution give them a good understanding of why this diet works.
Here are some other good resources and sites that collect links to studies and other helpful information:
One of the originators of modern day Paleo eating; great research links:
Scientific info., eating Paleo for Autoimmune diseases, and more women friendly
This one is good for medical info. and more guy friendly:
Great recipes and tips for kids
Simple and well tested recipes (esp. for baked goods)
The Paleo Solution, Robb Wolf
The Paleo Diet, Loren Cordain
Eat Like a Dinosaur (by Paleo Parents)
Paleo Cooking by Elana
Grain Brain - David Perlmutter
Paleo Magazine - very well done and interesting magazine about all things Paleo - really nice way to learn even more about healthy living and products