Aragorn bestowed upon you the authority to speak on his behalf, and you have gone with Yadêphal to hear what his master, once the Mouth of Sauron, has to say to Gondor.
Yadêphal is in Barad-dûr, where Sauron reigned.
Aragorn has given you authority to treat with Dulgabêth on his behalf.
Dulgabêth is in Barad-dûr, and demands to know why you have come to treat with him instead of Aragorn.
Aragorn has given you authority to hear what Dulgabêth has to say.
Aragorn will never accept this offer, but you should tell Dulgabêth of your intention to bring it before the king.