Google Indexed URLs Going Down?

I have added a sitemap to my Google Webmaster Tools with 36 urls. Every time it indexes my site again, the number of indexed urls goes down! A month ago I had 21 out of the 36 web pages indexed and I just figured google hadn't got to the rest yet. But now it says I only have 12 urls indexed. What the heck. Any ideas? Should I resubmit my sitemap?