How to rate-limit in nginx, but including/excluding certain IP addresses?

I'm able to use limit_req to rate-limit all requests to my server. However I'd like to remove the rate restriction for certain IP addresses (i.e. whitelist) and use a different rate restriction for certain others (i.e. certain IPs I'd like as low as 1r/s). I tried using conditionals (e.g. if ( $remote_addr = "1.2.3.4" ) {}) but that seems to work only with rewrite rules, not for rate-limit rules.

[halfrack]

limit_req_zone の key に map variable を使うと柔軟に制御出来る。 geo と map をネストする理由が無念。