In their opening two league defeats by St Johnstone and Celtic, Kilmarnock shipped nine goals and were alarmingly open defensively.
That trend continued as Ayr started on the front foot, with striking duo Taylor and Stirton allowed acres of space in which to operate.
Scott McMann and Reece Lyon both shot just wide after Stirton's opener and the visitors deservedly led with half-time approaching.
McCann turned to his bench early, bringing winger Clescenco on for right-back Jamie Brandon after just 39 minutes and his side got on the scoresheet soon afterwards.
They again started slowly in the second half and Ayr - who were unbeaten in all competitions this term before their trip up the road - found joy in transition throughout.
Despite their defensive flaws, Kilmarnock stayed in the contest and started to turn the screw as the visitors tired.
Stone made good saves to deny Robbie Deas and Tom Lowery, while Mark O'Hara headed wide from close range, but eventually the pressure told as Schjonning-Larsen drove forward and beat Stone at his near post.
The on-loan Heart of Midlothian goalkeeper was again called into action as he kept out Clesenco, but his first mistake of the game proved costly.
He came off his line to cleanly gather a cross from the left, but as he landed, the ball came loose and Clesenco kept his cool amid the derby chaos to win it.
It was far from a vintage Kilmarnock performance, but after two chastening league defeats, it was the result that mattered most for McCann.
