James Kugel's book "Great Poems of the Bible" provides great insight into how the Psalms might have been understood at the time of their writing. The podcasts in this series rely heavily on his work.
Psalm 29, Lebanon Skips Like a Calf
In the pagan world, the phenomena of nature were regarded as manifestations of celestial bureaucrats, one for storms, another over the sea, yet another for fertility, etc. In Israel, God was over all but within none. He is the cause of everything and shares His glory with none other.
Psalm 42, As the Deer
Biblically, the soul is often presented as both part of and apart from a man. In this, it is likened to a man marrying the daughter of the king. While she and he are one flesh, she still speaks to her father about things going on in their household. Similarly, as her husband the man is responsible for her care and well being. Finally, although he knows her intimately, there are things going on in her head and heart that he will never understand.
Psalm 51, A Pure Heart
David had committed adultery with Bathsheba and arranged for her husband to be killed in battle. Upon being confronted by Nathan the prophet and realizing his sin, David pleaded with God to accept his repentance and restore his heart to right relationship with his Lord.