Map and Remove nil values in Ruby

295

42

I have a map which either changes a value or sets it to nil. I then want to remove the nil entries from the list. The list doesn't need to be kept.

This is what I currently have:

items.map! { |x| process_x url } # [1, 2, 3, 4, 5] => [1, nil, 3, nil, nil]
items.select! { |x| !x.nil? } # [1, nil, 3, nil, nil] => [1, 3]

I'm aware I could just do a loop and conditionally collect in another array like this:

new_items = []
items.each do |x|
    x = process_x x
    new_items.append(x) unless x.nil?
end
items = new_items

But it doesn't seem that ruby-esque. Is there a nice way to run a function over a list removing/excluding the nils as you go?

Peter Hamilton

Posted 2012-11-21T02:29:13.470

Reputation: 3 164

Answers

786

You could use compact:

[1, nil, 3, nil, nil].compact
=> [1, 3] 

I'd like to remind people that if you're getting an array containing nils as the output of a map block, and that block tries to conditionally return values, then you've got code smell and need to rethink your logic.

For instance, if you're doing something that does this:

[1,2,3].map{ |i|
  if i % 2 == 0
    i
  end
}
# => [nil, 2, nil]

Then don't. Instead, prior to the map, reject the stuff you don't want or select what you do want:

[1,2,3].select{ |i| i % 2 == 0 }.map{ |i|
  i
}
# => [2]

I consider using compact to clean up a mess as a last-ditch effort to get rid of things we didn't handle correctly, usually because we didn't know what was coming at us. We should always know what sort of data is being thrown around in our program; Unexpected/unknown data is bad. Anytime I see nils in an array I'm working on, I dig into why they exist, and see if I can improve the code generating the array, rather than allow Ruby to waste time and memory generating nils then sifting through the array to remove them later.

'Just my $%0.2f.' % [2.to_f/100]

the Tin Man

Posted 2012-11-21T02:29:13.470

Reputation: 133 358

7Doh! Thanks, just having a late-night moment... – Peter Hamilton – 2012-11-21T02:32:31.330

20Now that's ruby-esque! – Christophe Marois – 2013-05-16T20:56:31.363

1Note: Doesn't filter out "" – FloatingRock – 2014-08-14T08:21:30.783

2Why should it? The OP needs to strip nil entries, not empty strings. BTW, nil isn't the same as an empty-string. – the Tin Man – 2014-08-14T19:31:34.270

7Both solutions iterate twice over the collection... why not use reduce or inject? – Ziggy – 2015-03-04T18:34:02.673

3It doesn't sound like you read the OPs question or the answer. The question is, how to remove nils from an array. compact is fastest but actually writing the code correctly in the start removes the need to deal with nils completely. – the Tin Man – 2015-03-04T21:27:27.803

2I disagree! The question is "Map and remove nil values". Well, to map and remove nil values is to reduce. In their example, the OP maps and then select out the nils. Calling map and then compact, or select and then map, amounts to making the same mistake: as you point out in your answer, it is a code smell. – Ziggy – 2015-08-19T03:52:31.297

2Answer from @Ziggy should be accepted as a correct answer – Anton – 2015-11-10T05:29:36.053

speaking to @Ziggy's suggestion, there is a page here demonstrating how to use inject or reduce http://www.potstuck.com/2011/07/25/map-if-in-ruby-and-an-introduction-to-rubys-inject/

– David West – 2016-03-15T17:03:33.387

The example is too simple: if the map were a match against a regex with capture groups, then running the select simply to choose the strings that matched, and then running the map to re-match for the groups is wasteful, and map.compact is exactly right. The answers to the question should have tackled the non-trival example. – android.weasel – 2017-08-23T20:20:36.890

Besides the point, but in this answer [1,2,3].select{ |i| i % 2 == 0 } can even be improved into [1,2,3].select(&:even?) – Jochem Schulenklopper – 2017-12-07T10:36:28.990

1@theTinMan why does the solution with select need another map? [1,2,3].select{ |i| i % 2 == 0 } is enough to return [2], right? The map part doesnt make sense as it returns the element itself map { |x| x } which equivalent to the input array. – rubyprince – 2017-12-19T11:53:03.920

@rubyprince that's exactly what I'm asking myself right now. I think it's just a mistake of the poster. You could easily remove map and the code behaves the same way at least as far as my knowledge of ruby, and the quick tests I did showed. – Helsing – 2018-10-05T19:02:37.487

@Helsing Looking at it again, I think what he is saying is instead of using map and compact like this [1, 2, 3].map { |i| i * 2 if i % 2 == 0 }.compact,it is better to use select and map instead [1, 2, 3].select { |i| i % 2 == 0 }.map { |i| i * 2 } to make the intent clearer – rubyprince – 2018-10-06T06:36:54.253

67

Try using #reduce or #inject!

[1, 2, 3].reduce([]) { |memo, i|
  if i % 2 == 0
    memo << i
  end

  memo
}

I agree with the accepted answer that we shouldn't map and compact, but not for the same reasons!

I feel deep inside that map-then-compact is equivalent to select-then-map. Consider: a map is a one-to-one function. If you are mapping from some set of values, and you map, then you want one value in the output set for each value in the input set. If you are having to select before-hand, then you probably don't want a map on the set. If you are having to select afterwards (or compact) then you probably don't want a map on the set. In either case you are iterating twice over the entire set, when a reduce only needs to go once.

Also, in English, you are trying to "reduce a set of integers into a set of even integers".

Ziggy

Posted 2012-11-21T02:29:13.470

Reputation: 9 975

2Poor Ziggy, no love for your suggestion. lol. plus one, someone else has hundreds of upvotes! – DDDD – 2015-03-25T16:05:38.297

2I believe that one day, with your help, this answer will surpass the accepted on. ^o^// – Ziggy – 2015-03-25T20:35:12.920

1+1 the currently accepted answer doesn't allow you to use the results of operations you performed during the select phase – chees – 2015-07-23T01:24:20.060

1reduce is the better solution. vote Ziggy! – David West – 2016-03-15T17:04:36.860

1iterating over enumerable datastructures twice if only on pass is needed like in the accepted answer seems wasteful. Thus reduce the number of passes by using reduce! Thanks @Ziggy – SSchneid – 2018-06-20T12:18:09.020

That's true! But doing two passes over a collection of n elements is still O(n). Unless your collection is so big that it doesn't fit in your cache, doing two passes is probably fine (I just think this is more elegant, expressive, and less likely to lead to bugs in the future when, say, the loops fall out of sync). If you like doing things in one pass too, you might be interested in learning about transducers! https://github.com/cognitect-labs/transducers-ruby

– Ziggy – 2018-06-20T21:34:29.317

34

In your example:

items.map! { |x| process_x url } # [1, 2, 3, 4, 5] => [1, nil, 3, nil, nil]

it does not look like the values have changed other than being replaced with nil. If that is the case, then:

items.select{|x| process_x url}

will suffice.

sawa

Posted 2012-11-21T02:29:13.470

Reputation: 128 390

24

If you wanted a looser criterion for rejection, for example, to reject empty strings as well as nil, you could use:

[1, nil, 3, 0, ''].reject(&:blank?)
 => [1, 3, 0] 

If you wanted to go further and reject zero values (or apply more complex logic to the process), you could pass a block to reject:

[1, nil, 3, 0, ''].reject do |value| value.blank? || value==0 end
 => [1, 3]

[1, nil, 3, 0, '', 1000].reject do |value| value.blank? || value==0 || value>10 end
 => [1, 3]

Fred Willmore

Posted 2012-11-21T02:29:13.470

Reputation: 2 437

4.blank? is only available in rails. – ewalk – 2014-09-18T21:23:08.670

For future reference, since blank? is only available in rails, we could use items.reject!(&amp;:nil?) # [1, nil, 3, nil, nil] =&gt; [1, 3] which is not coupled to rails. (wouldn't exclude empty strings or 0s though) – Fotis – 2017-08-07T09:40:10.760

20

@the Tin Man, nice - I din't know this method. Well, definitely compact is the best way, but still can be also done with simple substraction:

[1, nil, 3, nil, nil] - [nil]
 => [1, 3]

Evgenia Manolova

Posted 2012-11-21T02:29:13.470

Reputation: 1 693

3Yes, set subtraction will work, but it's about half as fast due to its overhead. – the Tin Man – 2014-01-16T18:44:30.293

2

each_with_object is probably the cleanest way to go here:

new_items = items.each_with_object([]) do |x, memo|
    ret = process_x(x)
    memo << ret unless ret.nil?
end

In my opinion, each_with_object is better than inject/reduce in conditional cases because you don't have to worry about the return value of the block.

pnomolos

Posted 2012-11-21T02:29:13.470

Reputation: 649

-1

One more way to accomplish it will be as shown below. Here, we use Enumerable#each_with_object to collect values, and make use of Object#tap to get rid of temporary variable that is otherwise needed for nil check on result of process_x method.

items.each_with_object([]) {|x, obj| (process x).tap {|r| obj << r unless r.nil?}}

Complete example for illustration:

items = [1,2,3,4,5]
def process x
    rand(10) > 5 ? nil : x
end

items.each_with_object([]) {|x, obj| (process x).tap {|r| obj << r unless r.nil?}}

Alternate approach:

By looking at the method you are calling process_x url, it is not clear what is the purpose of input x in that method. If I assume that you are going to process the value of x by passing it some url and determine which of the xs really get processed into valid non-nil results - then, may be Enumerabble.group_by is a better option than Enumerable#map.

h = items.group_by {|x| (process x).nil? ? "Bad" : "Good"}
#=> {"Bad"=>[1, 2], "Good"=>[3, 4, 5]}

h["Good"]
#=> [3,4,5]

Wand Maker

Posted 2012-11-21T02:29:13.470

Reputation: 14 621