Don't replace table.remove(f, k), it's ok.
table.remove is meant to be used with tables with "valid indices." Your top-level table has "holes" and non-integer keys, whereas your deeper tables have "valid indices."
The other issue is that you are modifying tables while you are iterating over them. The combination of table.remove for a "list" iterated over with ipairs, and setting to nil with pairs seems to be ok, but not other combinations.
ipairs, remove with table.remove
pairs, remove with t[k] = nil