Suggestions for good NLP corpora

I'm trying to find a good NLP corpus in order to learn the grammatical structures from sentences. Specifically, I'm doing completely supervised training by using the annotated parse trees. The testing procedure involves re-creating those same parse trees from a given sentence.

So far I've tested and trained on a toy grammar (with good success), and then stepped up to using NLTK's subset of the Penn Treebank. While I'm getting promising results from the Penn Treebank, I think the small size (5% of the total) is responsible for at least some of the error.

While I have the option to buy the rest of the Penn Treebank, it's not clear from their description if the other 95% contains the same annotations. Also, I'd rather avoid paying if there's already good data out there for free.

One option would be to use the Stanford Parser to create the annotations for me (I'm more interested in learning the rules than specifically human-annotated text). While I have code now to do this for me, I'm having trouble finding a good source of sentences.

The books from Project Gutenberg appear to in decent shape (I think there's still some filtering that has to happen -- I come across random things like 'underlined' words and 'random-hypthenations'. Additionally, because of the nature of the project, many of the books are older and contain much more complex sentences than I would like. For example, I tried two of Jane Austen's books (they are relatively large in size), but after examining their parse trees have decided to try to find something simpler.

I've also (briefly) looked at Wikipedia, but it looks like non-words will often appear in the text as well as other types of errors. Again, this can be filtered, but if this has already been done, I'd much rather use someone else's work.

So my general question is: Can anyone suggest a good source that has relatively simple sentence construction and is fairly clean?

What corpora are other people using? I tried doing a basic literature search, but I'm having trouble finding any general corpora used.

edit oops, I'm an idiot. The Penn Treebank document does say the number of skeletal parses it contains:

                             Tagged for      Skeletal
 Description                Part-of-Speech  Parsing 
                            (Tokens)        (Tokens)

Dept. of Energy abstract          231,404   231,404  
Dow Jones Newswire stories      3,065,776 1,061,166         
Dept. of Agriculture bulletins     78,555    78,555 
Library of America texts          105,652   105,652        
MUC-3 messages                    111,828   111,828          
IBM Manual sentences               89,121    89,121          
WBUR radio transcripts             11,589    11,589 
ATIS sentences                     19,832    19,832          
Brown Corpus, retagged          1,172,041 1,172,041

Total:                          4,885,798 2,881,188