And shout for joy, all you who are upright in heart.
David finishes his celebration of repentance with a call for all repentant sinners to take stock of what God has done for them. He highlights the new thinking under which they now operate. This new thinking can be seen in their perspectives on righteousness, their own hearts, and God.
Repentant sinners know a different kind of righteousness
David addresses this verse to "righteous ones." Of whom does he speak? Our cultural conceptions of righteousness present some peculiar distortions. In North America, it is commonly believed that a "righteous one" is a person who is generally virtuous, is active in church, does not sin in overt ways or follows a strict moral code (also, he is boring, uptight, angry and out of touch). Biblical repentance knocks the stuffing out of that view. A person who fits the cultural definition has no need of repentance, he is probably too "good" for it. But the outward goodness man can achieve is very different from the righteousness God requires. What is the Biblical concept?

True righteousness moves from confession of sin to an acknowledgement of God's perfect goodness. In and of Himself, He is pure, He is holy, He is good, He is the measure of goodness and truth. To make the point very clear to us, God has revealed His transcendent perfection through creation. But He didn't stop there - He also sent us the living epitome of righteousness in human form through the incarnation of Christ.

For believers, the outward acts of righteousness which form a righteous character are built upon this inner righteousness. They are the fruit of the righteousness which comes to them as a gift from God. Any other kind of outward righteousness is false and useless, as every repentant sinner knows.
Repentant sinners see the significance of the heart
David recognizes that his sinful choices were the natural outflow of a depraved heart. He has learned something new about himself: he is sinful to the core. Now he is much more aware of the desires of his heart, much more grateful for any evidence of grace, and much more deliberate about rooting out evil thinking.
When we see our hearts as the idol factories that they are, we stop believing our own selfish thinking and no longer trust ourselves. Instead, we believe the testimony of the Word of God. We give serious consideration to the advice of others who see us more objectively than we see ourselves. We cry out to God for mercy every day - knowing many seeds of wickedness lie within us. More than that: we confess that we don't even realize the seeds of wickedness that lie dormant in our hearts.
We also realize that the transforming work of God happens in our hearts as He creates new desires within us. All our desire for good is evidence of His inner work of grace.
Consider the meaning of the Hebrew word for upright:
Upright = Heb YASHAR - straight, level, smooth, by impl. pleasing.
YASHAR can describe a road that is clear and unobstructed, hence smooth. In the context of this psalm, the connotation is that God is pleased when our hearts have a clear, unobstructed path to Him. In other words, we must reveal our hearts to Him, leaving nothing hidden or unconfessed. We must keep open communication with Him. We must bring down every high thing (such as our proud, defensive thinking), and raise up every low place (our hidden deeds). In this way we will maintain an intimate communion with the Father, which is an unfailing spring of delight.
Repentant sinners find overwhelming joy in God
All joy has a cause, and David calls us to rejoice for the very best reason: God Himself. Earlier in the psalm God was pressing a firm hand down on us, bringing us to a barren and desolate place of intense thirst. He seemed to be terrorizing our consciences, and we found no joy in Him. But now He becomes the ground of our gladness, the reason for our rejoicing, the basis of our blessedness.
Here are a few reasons why you will see repentant sinners rejoicing with unbridled enthusiasm:
~ God was patient and did not give us what we deserve. While we sinned, He granted time for repentance.
~ God completely and freely forgave us for all the ways we sinned in act, word and thought.
~ God covered our sinful nature by imputing Christ's righteousness to us.
~ God made atonement for our sin by offering a perfect sacrifice in our place.
~ God proved the sufficiency of the atonement by raising our perfect representative from the dead, thereby guaranteeing our forgiveness and acceptance before Him.
~ God granted us renewed fellowship with Himself.
~ God accepts us exactly as we are.
~ God loves us too much to leave us as we are. He promises to change us as we walk with Him.
~ God has promised to help us in our ongoing struggle against sin. He has given us a great High Priest Who offers continued forgiveness and help.
~ As we struggle with the ongoing destruction of a sinful world, the spiritual powers of evil that rule here, and the corruptions of our own hearts, God remains the stable and steadfast ground of our hope, a changeless fountain of grace, our wise Guide, our caring Shepherd, our loving Father and our faithful Friend.

Dear friend, rest your weary soul in His everlasting grace today.