If you want to apply a filter on data, you need to attach that to the Permission Set. If you need to create wide ranging filters for data permissions, then it takes a lot of maintenance to apply multiple filters across different user permission sets (and no real certainty if any are conflicting across Sets)

For example, if you wanted to remove access to sub-contractor Vendors (maybe as defined in the Vendor Posting Group) for all staff other than Accounts you would have to create the same table filters for each Permission Set various users belonged to (and that would be across several tables). If you wanted different people to see different types of Vendors, then that gets exponentially more hassle.

It would be much simpler for a "Security Filter Set" to be created independently of a Permission Set and be applied directly to the User (and / or Permission Set). If both wee applied, then the combination of both filters should be applied.

It would be even more beneficial if filters could be inherited. In teh sub-contractor example, it would be better to restrict access to vendors by filtering on Vendor Posting Group and that filter then being applied to Vendor Ledger Entries, General Ledger Entries, Invoices, Posted Invoices, Credits, etc. Rather than having to specify the same filters across all these tables manually (if the tables you want contains the identifying value (like Vendor Posting Group)).

This way you can create and maintain fewer Permission Sets and allow for faster ways to set up data restrictions in the business.