You may be right, or it may be CYA from the Browns. But even assuming you're correct that it was the Browns' decision, Hue already was on record saying that he would like to coach the Senior Bowl, and the announcement that the Browns wouldn't be doing that coaching didn't come from him. So it pretty clearly was a decision made by the new GM against Hue's own wishes, and people who follow the league know that. So there are two possible narratives: 1) Hue is so incompetent that the league wouldn't let him coach the Senior Bowl, or 2) the Browns GM refused to let Hue coach the Senior Bowl despite the league's invitation, and despite Hue's own publicly-expressed wishes in that regard. Either way, it is a really bad look for the Browns.
Also, if it was the Browns' decision, I think it was probably a different reason than a desire for Hue not to become attached to certain players. After all, if Dorsey is making the draft-day decisions, he's going to pick who he wants, regardless of any attachment Hue may have developed.
I think it more likely that (assuming it was the Browns' decision) Dorsey viewed Hue coaching the Senior Bowl as a waste of time that a 1-31 coach cannot excuse. In other words "focus on your own damn job of making this team better." In a sense, it's a way Dorsey can punish Hue given that Jimmy has otherwise tied his hands.
In any case...toxic as hell.