Skip to content

Commit 9fbaf7e

Browse files
committed
Squashed 'json/' content from commit 29e2c7c
git-subtree-dir: json git-subtree-split: 29e2c7c
0 parents  commit 9fbaf7e

64 files changed

Lines changed: 4543 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
TODO

.travis.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
language: python
2+
python: "2.7"
3+
install: pip install jsonschema
4+
script: bin/jsonschema_suite check

LICENSE

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Copyright (c) 2012 Julian Berman
2+
3+
Permission is hereby granted, free of charge, to any person obtaining a copy
4+
of this software and associated documentation files (the "Software"), to deal
5+
in the Software without restriction, including without limitation the rights
6+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7+
copies of the Software, and to permit persons to whom the Software is
8+
furnished to do so, subject to the following conditions:
9+
10+
The above copyright notice and this permission notice shall be included in
11+
all copies or substantial portions of the Software.
12+
13+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19+
THE SOFTWARE.

README.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
JSON Schema Test Suite [![Build Status](https://travis-ci.org/json-schema/JSON-Schema-Test-Suite.png?branch=develop)](https://travis-ci.org/json-schema/JSON-Schema-Test-Suite)
2+
======================
3+
4+
This repository contains a set of JSON objects that implementors of JSON Schema
5+
validation libraries can use to test their validators.
6+
7+
It is meant to be language agnostic and should require only a JSON parser.
8+
9+
The conversion of the JSON objects into tests within your test framework of
10+
choice is still the job of the validator implementor.
11+
12+
Structure of a Test
13+
-------------------
14+
15+
If you're going to use this suite, you need to know how tests are laid out. The
16+
tests are contained in the `tests` directory at the root of this repository.
17+
18+
Inside that directory is a subdirectory for each draft or version of the
19+
schema. We'll use `draft3` as an example.
20+
21+
If you look inside the draft directory, there are a number of `.json` files,
22+
which logically group a set of test cases together. Often the grouping is by
23+
property under test, but not always, especially within optional test files
24+
(discussed below).
25+
26+
Inside each `.json` file is a single array containing objects. It's easiest to
27+
illustrate the structure of these with an example:
28+
29+
```json
30+
{
31+
"description": "the description of the test case",
32+
"schema": {"the schema that should" : "be validated against"},
33+
"tests": [
34+
{
35+
"description": "a specific test of a valid instance",
36+
"data": "the instance",
37+
"valid": true
38+
},
39+
{
40+
"description": "another specific test this time, invalid",
41+
"data": 15,
42+
"valid": false
43+
}
44+
]
45+
}
46+
```
47+
48+
So a description, a schema, and some tests, where tests is an array containing
49+
one or more objects with descriptions, data, and a boolean indicating whether
50+
they should be valid or invalid.
51+
52+
Coverage
53+
--------
54+
55+
Currently, draft 3 should have essentially full coverage for the core schema.
56+
57+
The beginnings of draft 4 are underway.
58+
59+
Who Uses the Test Suite
60+
-----------------------
61+
62+
This suite is being used by:
63+
64+
* [json-schema-validator (Java)](https://github.com/fge/json-schema-validator)
65+
* [jsonschema (python)](https://github.com/Julian/jsonschema)
66+
* [aeson-schema (haskell)](https://github.com/timjb/aeson-schema)
67+
* [direct-schema (javascript)](https://github.com/IreneKnapp/direct-schema)
68+
* [jsonschema (javascript)](https://github.com/tdegrunt/jsonschema)
69+
* [JaySchema (javascript)](https://github.com/natesilva/jayschema)
70+
* [jesse (Erlang)](https://github.com/klarna/jesse)
71+
72+
If you use it as well, please fork and send a pull request adding yourself to
73+
the list :).
74+
75+
Contributing
76+
------------
77+
78+
If you see something missing or incorrect, a pull request is most welcome!

bin/jsonschema_suite

Lines changed: 277 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,277 @@
1+
#! /usr/bin/env python
2+
import sys
3+
import textwrap
4+
5+
try:
6+
import argparse
7+
except ImportError:
8+
print textwrap.dedent("""
9+
The argparse library could not be imported. jsonschema_suite requires
10+
either Python 2.7 or for you to install argparse. You can do so by
11+
running `pip install argparse`, `easy_install argparse` or by
12+
downloading argparse and running `python2.6 setup.py install`.
13+
14+
See https://pypi.python.org/pypi/argparse for details.
15+
""".strip("\n"))
16+
sys.exit(1)
17+
18+
import errno
19+
import fnmatch
20+
import json
21+
import os
22+
import random
23+
import shutil
24+
import unittest
25+
import warnings
26+
27+
if getattr(unittest, "skipIf", None) is None:
28+
unittest.skipIf = lambda cond, msg : lambda fn : fn
29+
30+
try:
31+
import jsonschema
32+
except ImportError:
33+
jsonschema = None
34+
35+
36+
ROOT_DIR = os.path.join(os.path.dirname(__file__), os.pardir)
37+
SUITE_ROOT_DIR = os.path.join(ROOT_DIR, "tests")
38+
39+
REMOTES = {
40+
"integer.json": {"type": "integer"},
41+
"subSchemas.json": {
42+
"integer": {"type": "integer"},
43+
"refToInteger": {"$ref": "#/integer"},
44+
},
45+
"folder/folderInteger.json": {"type": "integer"}
46+
}
47+
REMOTES_DIR = os.path.join(ROOT_DIR, "remotes")
48+
49+
TESTSUITE_SCHEMA = {
50+
"$schema": "http://json-schema.org/draft-03/schema#",
51+
"type": "array",
52+
"items": {
53+
"type": "object",
54+
"properties": {
55+
"description": {"type": "string", "required": True},
56+
"schema": {"required": True},
57+
"tests": {
58+
"type": "array",
59+
"items": {
60+
"type": "object",
61+
"properties": {
62+
"description": {"type": "string", "required": True},
63+
"data": {"required": True},
64+
"valid": {"type": "boolean", "required": True}
65+
},
66+
"additionalProperties": False
67+
},
68+
"minItems": 1
69+
}
70+
},
71+
"additionalProperties": False,
72+
"minItems": 1
73+
}
74+
}
75+
76+
77+
def files(paths):
78+
for path in paths:
79+
with open(path) as test_file:
80+
yield json.load(test_file)
81+
82+
83+
def groups(paths):
84+
for test_file in files(paths):
85+
for group in test_file:
86+
yield group
87+
88+
89+
def cases(paths):
90+
for test_group in groups(paths):
91+
for test in test_group["tests"]:
92+
test["schema"] = test_group["schema"]
93+
yield test
94+
95+
96+
def collect(root_dir):
97+
for root, dirs, files in os.walk(root_dir):
98+
for filename in fnmatch.filter(files, "*.json"):
99+
yield os.path.join(root, filename)
100+
101+
102+
class SanityTests(unittest.TestCase):
103+
@classmethod
104+
def setUpClass(cls):
105+
print "Looking for tests in %s" % SUITE_ROOT_DIR
106+
cls.test_files = list(collect(SUITE_ROOT_DIR))
107+
print "Found %s test files" % len(cls.test_files)
108+
assert cls.test_files, "Didn't find the test files!"
109+
110+
def test_all_files_are_valid_json(self):
111+
for path in self.test_files:
112+
with open(path) as test_file:
113+
try:
114+
json.load(test_file)
115+
except ValueError as error:
116+
self.fail("%s contains invalid JSON (%s)" % (path, error))
117+
118+
def test_all_descriptions_have_reasonable_length(self):
119+
for case in cases(self.test_files):
120+
descript = case["description"]
121+
self.assertLess(
122+
len(descript),
123+
60,
124+
"%r is too long! (keep it to less than 60 chars)" % (descript,)
125+
)
126+
127+
def test_all_descriptions_are_unique(self):
128+
for group in groups(self.test_files):
129+
descriptions = set(test["description"] for test in group["tests"])
130+
self.assertEqual(
131+
len(descriptions),
132+
len(group["tests"]),
133+
"%r contains a duplicate description" % (group,)
134+
)
135+
136+
@unittest.skipIf(jsonschema is None, "Validation library not present!")
137+
def test_all_schemas_are_valid(self):
138+
for schema in os.listdir(SUITE_ROOT_DIR):
139+
schema_validator = jsonschema.validators.get(schema)
140+
if schema_validator:
141+
test_files = collect(os.path.join(SUITE_ROOT_DIR, schema))
142+
for case in cases(test_files):
143+
try:
144+
schema_validator.check_schema(case["schema"])
145+
except jsonschema.SchemaError as error:
146+
self.fail("%s contains an invalid schema (%s)" %
147+
(case, error))
148+
else:
149+
warnings.warn("No schema validator for %s" % schema)
150+
151+
@unittest.skipIf(jsonschema is None, "Validation library not present!")
152+
def test_suites_are_valid(self):
153+
validator = jsonschema.Draft3Validator(TESTSUITE_SCHEMA)
154+
for tests in files(self.test_files):
155+
try:
156+
validator.validate(tests)
157+
except jsonschema.ValidationError as error:
158+
self.fail(str(error))
159+
160+
def test_remote_schemas_are_updated(self):
161+
for url, schema in REMOTES.iteritems():
162+
filepath = os.path.join(REMOTES_DIR, url)
163+
with open(filepath) as schema_file:
164+
self.assertEqual(json.load(schema_file), schema)
165+
166+
167+
def main(arguments):
168+
if arguments.command == "check":
169+
suite = unittest.TestLoader().loadTestsFromTestCase(SanityTests)
170+
result = unittest.TextTestRunner(verbosity=2).run(suite)
171+
sys.exit(not result.wasSuccessful())
172+
elif arguments.command == "flatten":
173+
selected_cases = [case for case in cases(collect(arguments.version))]
174+
175+
if arguments.randomize:
176+
random.shuffle(selected_cases)
177+
178+
json.dump(selected_cases, sys.stdout, indent=4, sort_keys=True)
179+
elif arguments.command == "remotes":
180+
json.dump(REMOTES, sys.stdout, indent=4, sort_keys=True)
181+
elif arguments.command == "dump_remotes":
182+
if arguments.update:
183+
shutil.rmtree(arguments.out_dir, ignore_errors=True)
184+
185+
try:
186+
os.makedirs(arguments.out_dir)
187+
except OSError as e:
188+
if e.errno == errno.EEXIST:
189+
print "%s already exists. Aborting." % arguments.out_dir
190+
sys.exit(1)
191+
raise
192+
193+
for url, schema in REMOTES.iteritems():
194+
filepath = os.path.join(arguments.out_dir, url)
195+
196+
try:
197+
os.makedirs(os.path.dirname(filepath))
198+
except OSError as e:
199+
if e.errno != errno.EEXIST:
200+
raise
201+
202+
with open(filepath, "wb") as out_file:
203+
json.dump(schema, out_file, indent=4, sort_keys=True)
204+
elif arguments.command == "serve":
205+
try:
206+
from flask import Flask, jsonify
207+
except ImportError:
208+
print textwrap.dedent("""
209+
The Flask library is required to serve the remote schemas.
210+
211+
You can install it by running `pip install Flask`.
212+
213+
Alternatively, see the `jsonschema_suite remotes` or
214+
`jsonschema_suite dump_remotes` commands to create static files
215+
that can be served with your own web server.
216+
""".strip("\n"))
217+
sys.exit(1)
218+
219+
app = Flask(__name__)
220+
221+
@app.route("/<path:path>")
222+
def serve_path(path):
223+
if path in REMOTES:
224+
return jsonify(REMOTES[path])
225+
return "Document does not exist.", 404
226+
227+
app.run(port=1234)
228+
229+
230+
parser = argparse.ArgumentParser(
231+
description="JSON Schema Test Suite utilities",
232+
)
233+
subparsers = parser.add_subparsers(help="utility commands", dest="command")
234+
235+
check = subparsers.add_parser("check", help="Sanity check the test suite.")
236+
237+
flatten = subparsers.add_parser(
238+
"flatten",
239+
help="Output a flattened file containing a selected version's test cases."
240+
)
241+
flatten.add_argument(
242+
"--randomize",
243+
action="store_true",
244+
help="Randomize the order of the outputted cases.",
245+
)
246+
flatten.add_argument(
247+
"version", help="The directory containing the version to output",
248+
)
249+
250+
remotes = subparsers.add_parser(
251+
"remotes",
252+
help="Output the expected URLs and their associated schemas for remote "
253+
"ref tests as a JSON object."
254+
)
255+
256+
dump_remotes = subparsers.add_parser(
257+
"dump_remotes", help="Dump the remote ref schemas into a file tree",
258+
)
259+
dump_remotes.add_argument(
260+
"--update",
261+
action="store_true",
262+
help="Update the remotes in an existing directory.",
263+
)
264+
dump_remotes.add_argument(
265+
"--out-dir",
266+
default=REMOTES_DIR,
267+
type=os.path.abspath,
268+
help="The output directory to create as the root of the file tree",
269+
)
270+
271+
serve = subparsers.add_parser(
272+
"serve",
273+
help="Start a webserver to serve schemas used by remote ref tests."
274+
)
275+
276+
if __name__ == "__main__":
277+
main(parser.parse_args())

remotes/folder/folderInteger.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"type": "integer"
3+
}

remotes/integer.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"type": "integer"
3+
}

0 commit comments

Comments
 (0)