Fresh allegations are again surrounding Prince Andrew, with a new book set for release next week containing further claims about the Duke of York’s private and public life. Entitled: The Rise and Fall of the House of York by Andrew Lownie revisits Andrew’s ties to convicted sex offender Jeffrey Epstein and even alleges he was involved in a physical altercation with Prince Harry after making comments about Meghan Markle. A spokesperson for Prince Harry told PEOPLE the claims were false, insisting there had never been a fight nor such remarks.

The book also delves into Andrew’s controversial relationship with alleged “Chinese spy” Yang Tengbo and the long-running “siege of Royal Lodge,” during which King Charles withdrew his brother’s £1 million-a-year personal allowance.

According to The Daily Telegraph’s royal correspondent Hannah Furness, palace officials are quietly discussing ways to further distance the Royal Family from Andrew. One rarely-used mechanism, she notes, would allow Parliament to remove not only a dukedom but even the title of “Prince,” though it would require legislation and Royal Assent.

Royal historian Christopher Wilson suggested that such action is more likely under Prince William, should he become king, describing him as “ruthless” about preserving the monarchy’s image in the social media era. By contrast, Wilson said King Charles has largely followed the late Queen Elizabeth II’s policy of avoiding overt family ruptures.

Andrew has already been stripped of his military titles and royal patronages in 2022 after facing a US civil lawsuit over sexual assault allegations — which he settled without admitting liability — and agreed to stop using his HRH title. Nevertheless, allegations have continued to surface. Questions remain about his decade as an international trade envoy, with claims that he maintained “dozens, if not hundreds” of questionable connections. Critics have noted that his overseas engagements often coincided with visits to ski resorts and golf courses, earning him the nickname “Air Miles Andy.”

His close relationship with his ex-wife, Sarah Ferguson, who still lives with him nearly 30 years after their divorce, has also attracted scrutiny. In 2010, she was recorded offering to sell access to Andrew to an undercover reporter.

Buckingham Palace declined to comment when approached about the new claims.