Brian Monetti, I restore Camaros
The quality and longevity of a cars paint job depend on three things. First, the prep work, including sanding, smoothing, and fully cleaning the surface. Second, the quality of the materials used, including paint and thinners. Third, the painting process itself.
Painting a car with a foam roller (not the fluffy ones that you use on drywall) is a low quality process compared to spray, but can still be pretty decent as long as you still put lots of effort into prep work and use good materials!
For prep, you need to sand the entire car. Even a single square inch missed will cause the paint to peel! You should sand to at least 400 grit, probably doing a once over with 220 and then working up to 400. Smooth any rough spots out and fill them with bondo as necessary. You also need to fully clean the car. A full water and soap wash (rinse all the soap!) followed by a wash with some mild thinners to remove any oils should do the job. You can keep it dust free by wiping with cheesecloth at the end. If you are doing any filling, take lots of care to properly smooth them and mix them properly. You will also need to prime the car. Do multiple coats and sand lightly between coats with 400 grit.
Now to the paint. I highly recommend not using Rustoleum. It does not have anti uv properties, and will fade in a year or two. It also will shrink over time in the sun and crack and peel. The paint is also too soft for automotive. What you really need is a quality paint that will cure and not shrink over time. An acrylic enamel is probably the best way to go for a beginner. It's a lot easier than basecoat-clearcoat. A paint that is made for automotive applications will last a lot longer and be better quality Also make sure you thin it properly with the right thinners. It will cost a bit more than Rustoleum but is well worth it.
If you paint with a foam roller, you will need to let each coat fully cure, and then sand between coats, using a 400 to 600 grit paper, just like the primer. Don't rush it!! If you try sanding paint that it still slightly wet you will really screw the paint job up. It will take a day or two between coats. Also don't press the roller too hard or the edges of the roller will bleed out and cause 'ropes' or lines in your paint. Paint lightly and take your time! When you are done with your last coat and it is fully cured, buff and wax the car. It will look great!!
Good luck and I hope this helps!