Learning to count in English can be difficult for language learners, especially since large numbers can become very complex. Not to mention the fact that English speakers from different regions say and pronounce some numbers totally differently!
To retain information long-term, you need to do more than just memorize. You need to truly understand the information.
First, like math, you can identify patterns that will make learning number sequences in English easier. Patterns add meaning and organization so you are not just memorizing blindly.
Second, recognize that, like most English words, numbers are pronounced differently by different people. Do not panic when someone else says a number in a way that is different from how you memorized it.