In the early days of his public life, Prince Charles was often dismissed as an earnest, but eccentric heir to the throne who was more interested in the arts and the environment than the traditional duties of a monarch. Behind palace doors, his projects were indulged but rarely respected, even by his mother Queen Elizabeth’s most senior aides. “The men in gray suits,” as Princess Diana once called them, have been the topic of frequent conversation lately as related to Prince Harry and his relationship with his father. As one royal author recently wrote, The King has had his own issues with palace advisors.

Christopher Wilson, author of The Windsor Knot, wrote in the Daily Mail that The King’s Trust—originally known as The Prince’s Trust—almost didn’t come to be until Prince Charles stood up to his mother’s senior aides. Per Wilson, the then-Prince of Wales was struggling to figure out his role as heir to the throne 50 years ago, knowing he’d likely have a long wait until he became King.

In the mid-1970s, young people in the U.K. were struggling with record levels of unemployment. After brainstorming with his great-uncle, Louis Mountbatten, Prince Charles came up with the “brilliant idea” to launch the Prince’s Trust. The charity would help give disadvantaged young people the skills and knowledge to find jobs, build character and “find a way out.” But similar to Prince Harry and Princess Diana, Prince Charles faced off with “a different generation of well-tailored nobodies who were determined he shouldn’t get his way,” per Wilson.

Wilson quoted from The King’s biographer, Jonathan Dimbleby, who described how “The Queen’s private secretary Sir Martin Charteris intervened” when the idea of the Prince’s Trust was presented. He felt there “could be a conflict of interest between the brand-new Prince’s Trust and other royal trusts established to do good.”

“The prevailing attitude among the Queen’s senior advisors was that the heir to the throne was still too young to be taken entirely seriously,” Dimbleby wrote. “He should be humoured—but not let off the Palace leash.” Prince Charles’s private secretary was thus advised to “Go steady on the Trust.”

However, the Prince of Wales would not be deterred. After securing funding, including £7,400 from his own Navy severance pay, he officially launched the Prince’s Trust in 1976. Through personal development programs, job-hunting help and support for entrepreneurs, Prince Charles developed the Prince’s Trust to help isolated young people move into work, education or training.

Charles’s persistence not only marked a turning point in his royal career, but for the future of royal philanthropy. Today, the organization is known as The King’s Trust and has helped more than a million young people from the ages of 16 to 30.