The timing of when you plant will vary depending on where you live. Basically, you can get started as soon as you're sure temperatures won't dip below freezing again. But don't worry about missing some magical deadline; you can also continue planting things throughout the spring and summer. When you're ready, here are the basic essentials you'll need:
1. POTS: The size can vary depending on what you're growing, but you want to make sure there's enough depth for the plants' roots to fully develop. You'll probably want something at least 8" in diameter; it's better to overestimate how much space you'll need than underestimate. Here are some general guidelines for picking the right pot size and material.
2. POTTING SOIL: You want a potting mix that's formulated for containers, not anything that says "garden soil." Use this calculator to figure out how many bags of soil you'll need based on the dimensions of your containers.
3. PLANTS: Farmers' markets are a great local source for herb and vegetable seedlings in the spring. Hardware and garden stores should have a decent selection as well.
A few other things that are nice to have, but not necessary: A trowel (for moving dirt around), a watering can (though you can also just use a large pitcher), and fertilizer to use once your plants are established (see item #14 in this post for more info).