If you Get Someone's Goat means to really irritate and annoy a person. The phrase originated in America in the early part of the 20th century. Goats have long been supposed to have a calming influence on horses and because of this were often the stable mates of highly-strung race horses, especially while being stabled at an unfamiliar race track. But, now and then, a fancied stallion's chance of success would be scuppered by dodgy opponents, who would slip into the stable and take the goat away. The racehorse would be nervy and perform badly in the race.