More Tempest Questions! Prospero's Magic Powers

I love when I research a play, I get to ask all these questions that fall into the "no clear answer" category and we get to discuss them. Here's my latest:

When, exactly, does Prospero gain his magical powers?

Are we to assume that he was actively studying magic before he was banished? Or that he was merely studying nature, etc... in general, and that only by the coincidence of landing on the magical island to his powers come to exist?

Here's my dilemma:

If we assume that he was already trying to become a mage, then it's a little weird that he just happens to land on an island populated by sprites (and once ruled by a witch) that magnifies his powers.

But if we assume that it was mostly the island that did all the magical heavy lifting, that seems like it would take a major edge off of all his "retirement" speeches and drowning his books and burying his staff. As if, by taking those things with him, he could have brought his magic back to Milan?

Bardfilm suggests the interpretation that leaving behind his books has more to do with "and now paying attention to being Duke like I should have done in the first place," but what about the staff? Prospero numerous times refers to his staff as an instrument of his powers. Breaking his staff clearly seems to indicate his voluntary choice to terminate his power.

What does everybody else think?