Skip to content

Commit 1fe05d9

Browse files
authored
Merge pull request #11 from rubyworks/feature/bundle-tapy-tapj
Bundle the TAP-Y and TAP-J reporters.
2 parents d6a60da + ac5bea3 commit 1fe05d9

8 files changed

Lines changed: 236 additions & 22 deletions

File tree

‎HISTORY.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,10 @@
88
the bare message.
99
* Bring summary, outline, and progress reporters into Rubytest and add a TAP 13
1010
reporter that counts completed results, including skipped and broken cases.
11-
* Remove CLI shortcuts for the unbundled TAP-Y and TAP-J formats.
11+
* Bring the TAP-Y and TAP-J reporters in from rubytest-tap, with their `-y` and
12+
`-j` shortcuts. The TAP-Y stream now opens with a YAML document (it was
13+
JSON), the TAP-J stream no longer ends with a `...` line, and both report
14+
skipped tests and cases.
1215
* Remove the old Recorder status callbacks and status-indexed result table.
1316
* Preserve test and teardown exceptions together in a result, and turn outcome
1417
hook failures into recorded errors.

‎README.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,9 +54,12 @@ Reporters implement `record(result)` for completed outcomes and
5454
available for formats that show nesting.
5555

5656
The built-in formats are `dotprogress` (default), `summary` (one status line per
57-
test), `outline` (nested cases), `progress` (running count and timing), and
58-
`tap` (TAP version 13). Select one with `rubytest -f FORMAT`. Formats load only
59-
when selected; TAP captures test output and emits it as diagnostics.
57+
test), `outline` (nested cases), `progress` (running count and timing), `tap`
58+
(TAP version 13), and `tapy` and `tapj` (TAP-Y and TAP-J, the structured
59+
streams that [tapout](https://github.com/rubyworks/tapout) reads). Select one
60+
with `rubytest -f FORMAT`; `-y` and `-j` are short for `-f tapy` and `-f tapj`.
61+
Formats load only when selected. The TAP formats capture test output: TAP
62+
emits it as diagnostics, and TAP-Y and TAP-J record it with each test.
6063

6164

6265
## Installation

‎demo/05_builtin_reporters.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,3 +73,46 @@ number actually recorded, including the broken and skipped cases.
7373
lines.grep(/# TODO /).size.assert == 1
7474
lines.grep(/\A# stdout: ok 999 - injected/).size.assert == 1
7575
lines.grep(/\Anot ok \d+ - broken case/).size.assert == 1
76+
77+
TAP-Y and TAP-J carry the same results as structured records: one for the
78+
suite, one for each case and test, and a final tally. TAP-Y is a stream of
79+
YAML documents, TAP-J a line of JSON for each record.
80+
81+
require 'yaml'
82+
require 'json'
83+
84+
readers = {
85+
'tapy' => lambda { |text| YAML.load_stream(text) },
86+
'tapj' => lambda { |text| text.lines.map { |line| JSON.parse(line) } }
87+
}
88+
89+
readers.each do |format, read|
90+
success, output, report = render.call(format)
91+
success.assert == false
92+
records = read.call(output)
93+
94+
records.first['type'].assert == 'suite'
95+
records.first['rev'].assert == 5
96+
records.last['type'].assert == 'final'
97+
records.last['counts']['total'].assert == report.total
98+
records.last['counts']['omit'].assert == 2
99+
100+
results = records.select { |record| record['status'] }
101+
results.size.assert == report.total
102+
results.map { |record| record['status'] }.sort.assert == %w[error fail omit omit pass todo]
103+
results.select { |record| record['status'] == 'omit' }.map { |record| record['type'] }.sort.assert == %w[case test]
104+
records.count { |record| record['type'] == 'case' && !record['status'] }.assert == 2
105+
end
106+
107+
A TAP-Y stream ends with the YAML end-of-document marker.
108+
109+
render.call('tapy')[1].lines.last.chomp.assert == '...'
110+
111+
The command line has a shortcut for each.
112+
113+
require 'rubytest/cli'
114+
{ '-y' => 'tapy', '--tapj' => 'tapj' }.each do |flag, format|
115+
cli = Test::CLI.new
116+
cli.options.parse!([flag])
117+
cli.format.assert == format
118+
end

‎lib/rubytest/cli.rb‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,12 @@ def options
6262
opt.on '-f', '--format NAME', 'report format' do |name|
6363
conf.format = name
6464
end
65+
opt.on '-y', '--tapy', 'shortcut for -f tapy' do
66+
conf.format = 'tapy'
67+
end
68+
opt.on '-j', '--tapj', 'shortcut for -f tapj' do
69+
conf.format = 'tapj'
70+
end
6571

6672
opt.on '-t', '--tag TAG', 'select tests by tag' do |tag|
6773
conf.tags.concat makelist(tag)

‎lib/rubytest/format/abstract_hash.rb‎

Lines changed: 23 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -83,26 +83,15 @@ def begin_test(test)
8383
# @return [Hash]
8484
#
8585
def skip_test(test, reason=nil)
86-
h = {}
87-
h['type' ] = 'test'
88-
h['status'] = 'omit'
89-
h['reason'] = reason if reason
90-
91-
merge_subtype h, test
92-
merge_setup h, test
93-
merge_label h, test
94-
#merge_comparison h, test, exception
95-
#merge_coverage h, test
96-
merge_source h, test
97-
#merge_exception h, test, exception
98-
merge_output h
99-
merge_time h
100-
101-
return h
86+
skip_record(test, reason)
10287
end
10388

89+
# A skipped case is reported like a skipped test, as a case.
90+
#
91+
# @return [Hash]
92+
#
10493
def skip_case(test_case, reason=nil)
105-
skip_test(test_case, reason).merge('type' => 'case')
94+
skip_record(test_case, reason).merge('type' => 'case')
10695
end
10796

10897
#
@@ -222,6 +211,23 @@ def end_suite(suite)
222211

223212
private
224213

214+
# The record of a skipped test or case.
215+
def skip_record(test, reason)
216+
h = {}
217+
h['type' ] = 'test'
218+
h['status'] = 'omit'
219+
h['reason'] = reason if reason
220+
221+
merge_subtype h, test
222+
merge_setup h, test
223+
merge_label h, test
224+
merge_source h, test
225+
merge_output h
226+
merge_time h
227+
228+
return h
229+
end
230+
225231
# For todo entries in particulr, i.e. NotImplementedError
226232
# exception, the return value represents the "todo" priority
227233
# level. The `Exception#priority` method returns an Integer

‎lib/rubytest/format/tapj.rb‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# encoding: UTF-8
2+
3+
module Test::Reporters
4+
5+
# TAP-J Reporter
6+
#
7+
# Writes the run as a TAP-J stream (revision 5): a line of JSON for
8+
# the suite, each case, each test and the final tally.
9+
#
10+
class Tapj < AbstractHash
11+
12+
REVISION = 5
13+
14+
#
15+
def initialize(runner, output: $stdout)
16+
require 'json'
17+
super(runner)
18+
@output = output
19+
end
20+
21+
#
22+
def begin_suite(suite)
23+
emit super(suite).merge('rev' => REVISION)
24+
end
25+
26+
#
27+
def begin_case(test_case)
28+
emit super(test_case)
29+
end
30+
31+
#
32+
def pass(test)
33+
emit super(test)
34+
end
35+
36+
#
37+
def fail(test, exception)
38+
emit super(test, exception)
39+
end
40+
41+
#
42+
def error(test, exception)
43+
emit super(test, exception)
44+
end
45+
46+
#
47+
def todo(test, exception)
48+
emit super(test, exception)
49+
end
50+
51+
#
52+
def skip_test(test, reason=nil)
53+
emit super(test, reason)
54+
end
55+
56+
#
57+
def skip_case(test_case, reason=nil)
58+
emit super(test_case, reason)
59+
end
60+
61+
#
62+
def end_suite(suite)
63+
emit super(suite)
64+
end
65+
66+
private
67+
68+
# Write one record to the stream.
69+
def emit(hash)
70+
@output.puts hash.to_json
71+
hash
72+
end
73+
74+
end
75+
76+
end

‎lib/rubytest/format/tapy.rb‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# encoding: UTF-8
2+
3+
module Test::Reporters
4+
5+
# TAP-Y Reporter
6+
#
7+
# Writes the run as a TAP-Y stream (revision 5): a YAML document for
8+
# the suite, each case, each test and the final tally.
9+
#
10+
class Tapy < AbstractHash
11+
12+
REVISION = 5
13+
14+
#
15+
def initialize(runner, output: $stdout)
16+
require 'yaml'
17+
super(runner)
18+
@output = output
19+
end
20+
21+
#
22+
def begin_suite(suite)
23+
emit super(suite).merge('rev' => REVISION)
24+
end
25+
26+
#
27+
def begin_case(test_case)
28+
emit super(test_case)
29+
end
30+
31+
#
32+
def pass(test)
33+
emit super(test)
34+
end
35+
36+
#
37+
def fail(test, exception)
38+
emit super(test, exception)
39+
end
40+
41+
#
42+
def error(test, exception)
43+
emit super(test, exception)
44+
end
45+
46+
#
47+
def todo(test, exception)
48+
emit super(test, exception)
49+
end
50+
51+
#
52+
def skip_test(test, reason=nil)
53+
emit super(test, reason)
54+
end
55+
56+
#
57+
def skip_case(test_case, reason=nil)
58+
emit super(test_case, reason)
59+
end
60+
61+
#
62+
def end_suite(suite)
63+
emit super(suite)
64+
@output.puts '...'
65+
end
66+
67+
private
68+
69+
# Write one record to the stream.
70+
def emit(hash)
71+
@output.puts hash.to_yaml
72+
hash
73+
end
74+
75+
end
76+
77+
end

‎lib/rubytest/runner.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -433,7 +433,7 @@ def reporter_load(format)
433433
# TODO: Could use finder gem to look these up, but that's yet another dependency.
434434
#
435435
KNOWN_FORMATS = %w{
436-
dotprogress progress outline summary tap test
436+
dotprogress progress outline summary tap tapy tapj test
437437
}
438438

439439
# Returns a list of available report types.

0 commit comments

Comments
 (0)