Skip to content

rspec-queue does not fail if there are NameErrors in specs #100

@eliotsykes

Description

@eliotsykes

I've just encountered the following behaviour with the rspec runner of ci-queue:

  1. Modify a spec to reference an uninitialized constant so as to trigger a NameError, e.g. by referring to a non-existent class:
describe MyNonExistentClass do
 ...
end
  1. Run rspec-queue
  2. rspec-queue exits successfully.

I guessed wrongly that rspec-queue would fail with a non-zero exit code, but it exited with 0.

Bare rspec exits with 1 in the above situation.

I'm wondering if this is intended behaviour - perhaps its needed for retries to work?

The error message output is:

NameError: uninitialized constant MyNonExistentClass

Debugging shows that line 400 is where the 0 exit code is returned, because the NameError causes @world.non_example_failure to be true. Syntax errors appear to have the same result:

success = true
@configuration.reporter.report(examples_count) do |reporter|
@configuration.add_formatter(BuildStatusRecorder)
@configuration.with_suite_hooks do
break if @world.wants_to_quit
queue.poll do |example|
success &= example.run(QueueReporter.new(reporter, queue, example))
break if @world.wants_to_quit
end
end
end
return 0 if @world.non_example_failure
success ? 0 : @configuration.failure_exit_code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions