Thanks for the replies.
I think this is mainly my system not doing what was expected and also me forgetting the software and live monitoring options at that point. The USB signal wasn't working the first time. Main audio outputs of the Kemper were still going through the focusrite (albeit the gains were all the way down) and that was what I was hearing. I then unplugged the cables after I played for a while, resulting in massive confusion when I tried it again.
I assumed the USB signal would be audible though my Mac via software monitoring but I guess you can only do that through Logic. I'm currently on my business computer so going to move over to my studio side and try it all in there through Logic.
I was hoping there was a way to monitor the USB input through Mac OS though without having to use GarageBand or something like that.