General William Farrar "Baldy" Smith was a brilliant, brave, cantankerous, and not necessarily trustworthy West Pointer from Vermont who had a complicated relationship with Grant.
It came to a head in 1864, when Grant had secured Smith's promotion to major general but placed him under the command of Major General Benjamin Butler, a considerably less trustworthy character, militarily incompetent yet in some ways a brilliant and partly principled Democratic politician. And 1864 was a presidential election year, when Lincoln needed Butler's support, so was reluctant to replace him.
Smith frames Chapter Two of The Last Circle of Ulysses Grant, where the conversation touches on the circumstances of his removal from command. Smith's version blames Grant's drinking problem, which I think by the end of his life he was prone to maliciously exaggerate -- as did many others about other occasions of Grant's alleged drinking. It is nonsense, for example, to say Grant was drunk at the Battle of Shiloh, although some people did say so at the time, prompting Lincoln to defend the general he would not meet for another two years. Those stories tended to stick, and unfairly damage Grant's reputation up to this day, because there is some truth behind them. People like George McClellan had seen Grant drunk in the prewar Army, and so were inclined to believe unfounded rumors.
Still, I also think it is possible that Butler used an occasion when he and Smith witnessed Grant drinking to save his own job and get rid of Smith.
My own view of Grant's drinking is that it was only a problem when he was away from Julia and the children, not because she was nagging or otherwise riding herd, but because he was secure and happy in her presence, so not tempted to self-medicate in alcoholic excess.
That meant Grant's drinking problem was nonexistent for the last 20-odd years of his life. The incident involving Smith and Butler is about the last instance of it.