I know in most places, the septic has to be PERC'd for whatever size of house you will be building (1-100 rooms). You might check that first (It already has a septic system/field so it should be easy to get the number from the realtor).
I am curious about this as well, there's a few lots around me that are between 3-10 acres and around $100k... I'd love to buy one to hang onto for 10 years then build a house and get out of this townhouse community.
I am mostly curious about the loan situation... I thought typically banks wanted you to build something on it and not just hold it as land. Perhaps you could do owner financing?