If you want more Ultron and The Vision, read Avengers: Rage of Ultron
Rick Remender and artist Jerome Opeña's new graphic novel Rage of Ultron is a hugely ambitious book that delves into the bizarre generational conflict between The Vision, his "father" Ultron, and Ultron's creator, founding Avenger Hank Pym. (Yeah, it's not Tony Stark like in the movie.) It's a very emotional story, and focuses a lot on the way Pym's strange blend of guilt, remorse, and pride over creating Ultron has affected his life and standing among his fellow Avengers.
If you'd like a lot more Black Widow, get Black Widow: The Finely Woven Thread.
Everyone who feels frustrated by the lack of a Black Widow solo movie can take solace in Nathan Edmonson and Phil Noto's current Black Widow series. It's basically exactly what you'd want from a Widow movie, but in comic form – spy intrigue, stylish visuals, and a version of Natasha that's smart, cool, and ultra-confident.
If you're interested in Hawkeye, check out Hawkeye: My Life As A Weapon.
Matt Fraction and David Aja's offbeat Hawkeye series is one of the most innovative and acclaimed superhero comics of the past few years. Unlike a lot of Avengers comics, this one is rather low key – it's about the quieter missions and unexpected adventures Clint Barton and his protégé Kate Bishop get into when they're not with the Avengers. It's a very funny and formally adventurous comic, with Fraction and Aja pulling off stunts like having an entire issue told from the perspective of Hawkeye's dog.
If you want a lot more Captain America, get the Captain America: Winter Solider Ultimate Collection.
Ed Brubaker's long run on Captain America through most of the '00s is now widely considered to be the definitive take on the character, and for good reason. Brubaker and artists like Steve Epting and Butch Guice amped up the spy/thriller aspects of the character's mythos, and essentially set the template for the version of Cap that you see in the movies. There's a lot of Brubaker Cap books, but the best place to start is at the beginning, where he introduces The Winter Soldier.
If you want more Scarlet Witch, try Avengers: The Children's Crusade.
It's not easy to recommend Scarlet Witch comics – not because there aren't any good ones, but because almost all Scarlet Witch-centric stories are a minefield of confusing continuity. This book by writer Allan Heinberg and artist Jim Cheung does deal with the aftermath of a previous story in which a disturbed Wanda Maximoff wiped out most of the mutants on earth, but you can easily jump into without having read anything else. Children's Crusade is essentially a redemption story about Wanda coming to terms with what she's done, and discovering that maybe she wasn't entirely to blame for that atrocity.
If you need more Iron Man, invest in The Invincible Iron Man Omnibus, Vol.1.
What Ed Brubaker was to Captain America, Matt Fraction was for Iron Man. Fraction and artist Salvador Larocca's long run on The Invincible Iron Man, which launched around the same time as the movie series, is both an exciting superhero soap opera and a nuanced take on Tony Stark and his many enemies. The first year of Fraction and Larocca's run is collected in this omnibus, and features a compelling story arc in which Stark's leadership of S.H.I.E.L.D. ends in disgrace, and he's hunted down by H.A.M.M.E.R., an evil organization that has taken its place. It's a great "on the run" story that removes most of Stark's advantages and forces him to make huge sacrifices.
If you want more Falcon, try All-New Captain America: Hydra Ascendant.
The Captain America you see in the movies may be Steve Rogers, but these days the Cap in the comics is none other than Sam "Falcon" Wilson, who has stepped into the role after Rogers was drained of the Super Soldier Serum. The first volume of All New Captain America by Rick Remender and Stuart Immonen tells the story of Sam's first mission as Captain America, as he tries to get to the bottom of a massive Hydra conspiracy. It doesn't go well for him.
If you want to read more about Thor, go for Thor: God of Thunder Vol. 1: The God Butcher.
Jason Aaron's most recent Thor comics star a mysterious new female version of the character who has replaced the Odinson we all know and love. Those comics are great, but if you're looking for classic Thor action, it's best to start at the beginning with the first volume of Thor: God of Thunder. Aaron and artist Esad Ribic's take on Thor is suitably epic and follows his battle against the mysterious God Butcher across three different eras – the present day Thor, the brash young Thor, and an extremely old Thor in the future.
If you crave more Hulk, try The Incredible Hulk: Planet Hulk.
Greg Pak's run on Hulk in the '00s took a lot of risks, and pretty much all of them paid off. Planet Hulk is the beginning of a trilogy of Hulk epics in which he's exiled to outer space by the Avengers, and he essentially becomes a gladiator on a savage planet called Sakaar. It's a very bold storyline that pushes Hulk outside his usual trappings, and puts him in situations where he's not always the strongest guy around.