I'm no expert, but I don't find this logic correct. As a child we have the advantage of not knowing a language, that is what makes us fast learners. But as adults, it's impossible to “now know” a language again, we already have our mother language and since we can't ignore it or shut it down from our thought process, it's best to embrace it as a crutch to learn new languages. Also, the languages we already know help us memorise vocabulary much faster.
Usually the material for TOEFL, IELTS, and TOEIC is in English (I've seen some material in Japanese, but probably because of the difference in characters).
And when you start learning English you usually watch movies/shows with subtitles on your own language, then with English subtitles, and finally without subtitles. So it is part of a process, however - in my case - English movies/shows with English subtitles were a fast way to associate sounds to written words.