If we enable SKAdNetwork, do we lose IDFA visibility unless the user opts into being tracked via AppTrackingTransparency?

No, SKAdNetwork is a completely separate system that runs in parallel to the device-level data you've traditionally used to measure your ad campaigns.

In other words, SKAdNetwork and the usage of AppTrackingTransparency to get device-level ad insights are mutually exclusive. SKAdNetwork does not require opt-in under the ATT framework, and enabling SKAN in your app will not trigger the ATT modal or automatically limit your ability to access IDFAs (though you will likely want to consider both changes at the same time).