If you are experiencing high memory comsumption with Lucene 2.9.0/3.0.1 it could be due…

1. … to a recently reported and fixed bug in StandardTokenizer where JFlex generated code was expanding a buffer (zzBuffer) and never trimming it down
   
   
   Uploaded with plasq’s Skitch!

2. … to another recently reported and fixed bug where IndexWriter held references to Readers used in your Fields, (and if you have apache tika’s reader, those can take up a lot of space)
   
   
   Uploaded with plasq’s Skitch!