Your issue probably isn't related to having a non-sealed airbox. There's a ton of non-boxed aftermarket intakes running around out there with the factory cooling setup that don't experience anything like what you are.
It helps to understand theory of operation in this case with several pieces at play.
The vast majority of the heat that you're seeing is generated within the blower itself when that air gets compressed into boost. That would be your BSS3 temp at 266F. Then it passes through the intercooler and that heat is transferred into the fluid depending on the temp of that fluid. With the amount of heat that gets generated and transferred in, that fluid will heat up super quick so it needs to be replaced by fluid that isn't heated (you need good flow for that). With a stock system, this process of heat transfer, flow, and heat extraction through the secondary and primary heat exchangers is able to bring those 266F (or more!) temps back down to a reasonable 130F-160F range.
Likely what's happening is that you have a flow issue. The fluid isn't getting replaced by cooler fluid fast enough in the intercooler bricks. It heats up, stays hot, and the hotter the fluid, the less heat it'll be able to extract out of the air charge. Typical reasons that I commonly see are air in lines, flow restrictions due to routing either because of a pinch or bend or too many right angles creating additional head pressure, weak intercooler pump, etc.
Note how I haven't mentioned anything about how your chiller is operating. That's another piece but that piece seems to be working fine based on what you've stated so far. I'd focus more on how things are routed and flowing. You can also increase the factory pump speeds in the tune which may help if you haven't already done this.
