The decision is complex, and takes into account a number of different variables: the program to which you're applying, your command of the English language, and your academic strengths and weaknesses. Given the weight that these test results can have on your future, it's no small decision.
Graduate School vs. Business School
Many business schools have come to accept the GRE in recent years and some graduate schools accept the GMAT. The programs to which you plan on applying is a powerful determinant of which test is best. If all of your schools accept one test but do not all accept the other, the choice is already made for you. If both are universally accepted, then you will want to choose the test that plays to your strengths.
The choice is more obvious for students who are applying to graduate school. Liberal arts majors will usually have an easier time with the GRE verbal section than the GMAT verbal.
The math section of the GRE is considerably easier for a number of reasons — namely, it doesn't adapt by question, you can skip around, and you are allowed a calculator. Math and science majors regularly score close to perfect on the GRE math section for these reasons.
Finally, the GRE is the test most graduate school admissions personnel are familiar with, and so, all things being equal, you're better off applying with a strong GRE score.
Business school applicants face a tougher decision. B-schools are long accustomed to GMAT scores, and if you believe you can score equally well on either test, the GMAT is probably the better option. You should only opt for the GRE over the GMAT if you're able to score markedly higher on the GRE. Since the GMAT is harder, there are a number of students for whom this happens to be true.
Many international applicants to business school opt for the GMAT for a few reasons. Most importantly, the GMAT doesn't test vocabulary. The esoteric and frequently arcane words which appear on text completion and sentence equivalence questions can be a nightmare for the ESL crowd.
Secondly, sentence correction questions test rules of grammar and written English that international students have had to learn more recently, and in greater detail, than native speakers. Reading comprehension also makes up a smaller fraction of GMAT verbal than it does of GRE verbal, and this can be the hardest section for students whose command of English isn't the best.
Finally, many foreign-born students are accustomed to more rigorous math programs than are American students and thus are less likely to be dissuaded from taking the GMAT by a harder quantitative section.
Weaker Math Students
The main reason some students struggle to reach a competitive score on the GMAT, but are able to do so on the GRE, is that the GMAT quantitative section is more difficult. Beyond the format differences mentioned earlier, the GMAT tends to have more involved questions which frequently result in unfriendly decimal and radical answers. Frankly: it's meaner.
Also, data sufficiency has a way of haunting frustrated students in a way that quantitative comparison questions don't. It's the number one math-oriented complaint of new GMAT students that I hear.
The decision of which test to take should not be made lightly.
Do your research. Look at the 25th and 75th percentile scores, for both tests, of accepted applicants to the schools to which you plan on applying. Take a free test to see where you are: ETS's Power Prep II software for the GRE, and GMAC's GMAT Prep software are both available to download for free.
Carefully consider how your strengths and weaknesses mesh with the preferences of your preferred programs. Above all, whichever test you decide to prepare for, commit to your success on that test by studying as far in advance of the deadline as you are able.