Rick,
You can hide detail by right-clicking on the gray boxes in the Pivot Chart,
clicking "Group and Show Detail" and then "Hide detail." Showing it again can
be a pain. I ended up dragging it off the chart and putting it back again.
Hiding the detail in the columns is a lot easier, since you can just
double-click the column header to show/hide.
If that's important to you, maybe you can put the sorting field in the
columns instead of the rows.

Also, read the message I just posted about doing averages in Pivot Charts.
It's somewhat related and an answer to something I was struggling with.

-Jason