Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
66bde68
Change grades url to ngrok for testing
heratyian Mar 24, 2025
c08fbc3
Disable token regex validation for now
heratyian Mar 24, 2025
7e4f228
Comment out spec validation code for now.
heratyian Mar 25, 2025
cdb11b7
Refactor grade runner into services and utils and make submission url…
heratyian Apr 2, 2025
a1eed91
Remove extra config line in readme
heratyian Apr 3, 2025
c8de483
Implement spec_service so we can still override local specs if needed.
heratyian Apr 3, 2025
056579f
Upgrade bundler because DidYouMean::SPELL_CHECKERS is deprecated. Add…
heratyian Apr 3, 2025
9fc01e9
Require active support gem so we can call present? and update spec se…
heratyian Apr 3, 2025
999acba
Update note on Gemfile
heratyian Apr 3, 2025
c17c770
Increment version since this is a significant refactor
heratyian Apr 3, 2025
13fa918
Refactor token regex to a constant
heratyian Apr 3, 2025
3fcfcba
Remove unused requires.
heratyian Apr 3, 2025
7241296
Add back zip download function to spec service.
heratyian Apr 3, 2025
b5fe86c
Set required ruby version in gemspec
heratyian Apr 3, 2025
e10c7eb
Update juwelier and gemspec for 0.0.14
heratyian Apr 3, 2025
0a07f27
Fix newlines and formatting
heratyian Apr 3, 2025
228b0a3
Add more test coverage
heratyian Apr 4, 2025
b2d76b0
Remove rspec test suite for now
heratyian Apr 4, 2025
89c6c86
Merge pull request #1 from DPI-WE/config-submission-url
heratyian Apr 4, 2025
a8d336d
Update README.markdown
heratyian Aug 15, 2025
6dc2f18
Upgrade oj to 3.16 to silence warning: ostruct was loaded from the st…
heratyian Aug 18, 2025
a217a05
Refactor how taks are loaded. Now only need require 'grade_runner/tas…
heratyian Aug 18, 2025
a5dbf95
Fix issue with JsonOutputFormatter
heratyian Aug 18, 2025
e2a54db
Explicitly set spec directory so rspec can find tests to run.
heratyian Aug 19, 2025
2741c26
Add test helpers for rspec tests using grade_runner
heratyian Sep 8, 2025
78c4684
Add test_helpers to module
heratyian Sep 8, 2025
67eb5ea
Remove test helpers from default imports
heratyian Sep 8, 2025
0e3c579
Refactor some test helper methods
heratyian Sep 8, 2025
6ecefa3
Add helpers to run scripts with stdin
heratyian Sep 8, 2025
b2bbd22
Add helper method to strip comments from source
heratyian Sep 8, 2025
c239f8e
Restructure test helper methods
heratyian Sep 8, 2025
15682a0
Fix name conflict with run_script
heratyian Sep 8, 2025
ad6b02e
Remove alias
heratyian Sep 8, 2025
4cc9442
Refactor test helper method names
heratyian Sep 10, 2025
ff49d6d
Merge pull request #2 from dpi-tta/add-test-helpers
heratyian Sep 10, 2025
45de32c
Add more support for argv
heratyian Sep 11, 2025
8a856bc
Merge pull request #3 from dpi-tta/add-test-helpers
heratyian Sep 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
source "https://rubygems.org"
# Add dependencies required to use your gem here.
# Example:

gem "activesupport", ">= 2.3.5"
gem "oj", "~> 3.13.12"
gem 'oj', '~> 3.16', '>= 3.16.3'
gem "octokit", "~> 5.0"
gem "zip"
gem "faraday-retry", "~> 1.0.3"
gem "rake", "~> 13"
# Add dependencies to develop your gem here.
# Include everything needed to run rake, tests, features, etc.

group :development do
gem "rspec", "~> 3.5.0"
gem "rdoc", "~> 6.1"
gem "bundler", "~> 2.1.4"
gem "juwelier", "~> 2.1.0"
gem "bundler", "~> 2.6.7"
gem "juwelier", "~> 2.4.9"
gem "simplecov", ">= 0"
gem "pry", "~> 0"
gem "pry-byebug", "~> 3"
Expand Down
32 changes: 20 additions & 12 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ GEM
tzinfo (~> 2.0)
addressable (2.8.4)
public_suffix (>= 2.0.2, < 6.0)
bigdecimal (3.2.2)
binding_of_caller (1.0.0)
debug_inspector (>= 0.0.1)
builder (3.2.4)
Expand Down Expand Up @@ -56,17 +57,21 @@ GEM
i18n (1.14.1)
concurrent-ruby (~> 1.0)
interception (0.5)
juwelier (2.1.3)
juwelier (2.4.9)
builder
bundler (>= 1.13)
git (>= 1.2.5)
bundler
git
github_api
highline (>= 1.6.15)
nokogiri (>= 1.5.10)
highline
kamelcase (~> 0)
nokogiri
psych
rake
rdoc
semver
semver2
jwt (2.7.0)
kamelcase (0.0.2)
semver2 (~> 3)
method_source (1.0.0)
mini_portile2 (2.8.1)
minitest (5.19.0)
Expand All @@ -85,7 +90,10 @@ GEM
octokit (5.6.1)
faraday (>= 1, < 3)
sawyer (~> 0.9)
oj (3.13.23)
oj (3.16.11)
bigdecimal (>= 3.0)
ostruct (>= 0.2)
ostruct (0.6.3)
pry (0.14.1)
coderay (~> 1.1)
method_source (~> 1.0)
Expand Down Expand Up @@ -130,7 +138,7 @@ GEM
sawyer (0.9.2)
addressable (>= 2.3.5)
faraday (>= 0.17.3, < 3)
semver (1.0.1)
semver2 (3.4.2)
simplecov (0.21.2)
docile (~> 1.1)
simplecov-html (~> 0.11)
Expand All @@ -152,11 +160,11 @@ PLATFORMS

DEPENDENCIES
activesupport (>= 2.3.5)
bundler (~> 2.1.4)
bundler (~> 2.6.7)
faraday-retry (~> 1.0.3)
juwelier (~> 2.1.0)
juwelier (~> 2.4.9)
octokit (~> 5.0)
oj (~> 3.13.12)
oj (~> 3.16, >= 3.16.3)
pry (~> 0)
pry-byebug (~> 3)
pry-doc (~> 0)
Expand All @@ -170,4 +178,4 @@ DEPENDENCIES
zip

BUNDLED WITH
2.1.4
2.6.7
4 changes: 2 additions & 2 deletions README.markdown
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# grade_runner

A Ruby client for [firstdraft Grades](https://grades.firstdraft.com)

A Ruby client for [Grades](https://grades.dpi.dev)

## Installation

Expand Down Expand Up @@ -48,6 +47,7 @@ if Rails.env.development? || Rails.env.test?
GradeRunner.config do |config|
config.default_points = 1 # default 1
config.override_local_specs = false # default true
config.submission_url = "https://your-grading-server-url.com" # Set custom submission URL
end
end
```
Expand Down
3 changes: 3 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ Juwelier::Tasks.new do |gem|
gem.email = ["raghu@firstdraft.com", "jelani@firstdraft.com"]
gem.authors = ["Raghu Betina", "Jelani Woods"]

# Note: rspec tests do not yet support 3.4 (BigDecimal)
gem.required_ruby_version = Gem::Requirement.new(">= 2")

# dependencies defined in Gemfile
end
Juwelier::RubygemsDotOrgTasks.new
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.0.13
0.0.14
80 changes: 34 additions & 46 deletions grade_runner.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
# DO NOT EDIT THIS FILE DIRECTLY
# Instead, edit Juwelier::Tasks in Rakefile, and run 'rake gemspec'
# -*- encoding: utf-8 -*-
# stub: grade_runner 0.0.13 ruby lib
# stub: grade_runner 0.0.14 ruby lib

Gem::Specification.new do |s|
s.name = "grade_runner".freeze
s.version = "0.0.13"
s.version = "0.0.14"

s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
s.require_paths = ["lib".freeze]
s.authors = ["Raghu Betina".freeze, "Jelani Woods".freeze]
s.date = "2025-03-17"
s.date = "2025-08-18"
s.description = "This gem runs your RSpec test suite and posts the JSON output to grades.firstdraft.com.".freeze
s.email = ["raghu@firstdraft.com".freeze, "jelani@firstdraft.com".freeze]
s.extra_rdoc_files = [
Expand All @@ -29,57 +29,45 @@ Gem::Specification.new do |s|
"VERSION",
"grade_runner.gemspec",
"lib/grade_runner.rb",
"lib/grade_runner/formatters/hint_formatter.rb",
"lib/grade_runner/formatters/json_output_formatter.rb",
"lib/grade_runner/railtie.rb",
"lib/grade_runner/runner.rb",
"lib/tasks/grade.rake",
"lib/tasks/grade_runner.rake",
"lib/grade_runner/services/config_service.rb",
"lib/grade_runner/services/github_service.rb",
"lib/grade_runner/services/grade_service.rb",
"lib/grade_runner/services/spec_service.rb",
"lib/grade_runner/services/token_service.rb",
"lib/grade_runner/tasks.rb",
"lib/grade_runner/tasks/grade.rake",
"lib/grade_runner/tasks/grade_runner.rake",
"lib/grade_runner/utils/path_utils.rb",
"spec/spec_helper.rb"
]
s.homepage = "http://github.com/firstdraft/grade_runner".freeze
s.licenses = ["MIT".freeze]
s.rubygems_version = "3.1.6".freeze
s.required_ruby_version = Gem::Requirement.new(">= 2".freeze)
s.rubygems_version = "3.4.6".freeze
s.summary = "A Ruby client for [firstdraft Grades](https://grades.firstdraft.com)".freeze

if s.respond_to? :specification_version then
s.specification_version = 4
end
s.specification_version = 4

if s.respond_to? :add_runtime_dependency then
s.add_runtime_dependency(%q<activesupport>.freeze, [">= 2.3.5"])
s.add_runtime_dependency(%q<oj>.freeze, ["~> 3.13.12"])
s.add_runtime_dependency(%q<octokit>.freeze, ["~> 5.0"])
s.add_runtime_dependency(%q<zip>.freeze, [">= 0"])
s.add_runtime_dependency(%q<faraday-retry>.freeze, ["~> 1.0.3"])
s.add_runtime_dependency(%q<rake>.freeze, ["~> 13"])
s.add_development_dependency(%q<rspec>.freeze, ["~> 3.5.0"])
s.add_development_dependency(%q<rdoc>.freeze, ["~> 6.1"])
s.add_development_dependency(%q<bundler>.freeze, ["~> 2.1.4"])
s.add_development_dependency(%q<juwelier>.freeze, ["~> 2.1.0"])
s.add_development_dependency(%q<simplecov>.freeze, [">= 0"])
s.add_development_dependency(%q<pry>.freeze, ["~> 0"])
s.add_development_dependency(%q<pry-byebug>.freeze, ["~> 3"])
s.add_development_dependency(%q<pry-doc>.freeze, ["~> 0"])
s.add_development_dependency(%q<pry-remote>.freeze, ["~> 0"])
s.add_development_dependency(%q<pry-rescue>.freeze, ["~> 1"])
s.add_development_dependency(%q<pry-stack_explorer>.freeze, ["~> 0"])
else
s.add_dependency(%q<activesupport>.freeze, [">= 2.3.5"])
s.add_dependency(%q<oj>.freeze, ["~> 3.13.12"])
s.add_dependency(%q<octokit>.freeze, ["~> 5.0"])
s.add_dependency(%q<zip>.freeze, [">= 0"])
s.add_dependency(%q<faraday-retry>.freeze, ["~> 1.0.3"])
s.add_dependency(%q<rake>.freeze, ["~> 13"])
s.add_dependency(%q<rspec>.freeze, ["~> 3.5.0"])
s.add_dependency(%q<rdoc>.freeze, ["~> 6.1"])
s.add_dependency(%q<bundler>.freeze, ["~> 2.1.4"])
s.add_dependency(%q<juwelier>.freeze, ["~> 2.1.0"])
s.add_dependency(%q<simplecov>.freeze, [">= 0"])
s.add_dependency(%q<pry>.freeze, ["~> 0"])
s.add_dependency(%q<pry-byebug>.freeze, ["~> 3"])
s.add_dependency(%q<pry-doc>.freeze, ["~> 0"])
s.add_dependency(%q<pry-remote>.freeze, ["~> 0"])
s.add_dependency(%q<pry-rescue>.freeze, ["~> 1"])
s.add_dependency(%q<pry-stack_explorer>.freeze, ["~> 0"])
end
s.add_runtime_dependency(%q<activesupport>.freeze, [">= 2.3.5"])
s.add_runtime_dependency(%q<oj>.freeze, ["~> 3.16", ">= 3.16.3"])
s.add_runtime_dependency(%q<octokit>.freeze, ["~> 5.0"])
s.add_runtime_dependency(%q<zip>.freeze, [">= 0"])
s.add_runtime_dependency(%q<faraday-retry>.freeze, ["~> 1.0.3"])
s.add_runtime_dependency(%q<rake>.freeze, ["~> 13"])
s.add_development_dependency(%q<rspec>.freeze, ["~> 3.5.0"])
s.add_development_dependency(%q<rdoc>.freeze, ["~> 6.1"])
s.add_development_dependency(%q<bundler>.freeze, ["~> 2.6.7"])
s.add_development_dependency(%q<juwelier>.freeze, ["~> 2.4.9"])
s.add_development_dependency(%q<simplecov>.freeze, [">= 0"])
s.add_development_dependency(%q<pry>.freeze, ["~> 0"])
s.add_development_dependency(%q<pry-byebug>.freeze, ["~> 3"])
s.add_development_dependency(%q<pry-doc>.freeze, ["~> 0"])
s.add_development_dependency(%q<pry-remote>.freeze, ["~> 0"])
s.add_development_dependency(%q<pry-rescue>.freeze, ["~> 1"])
s.add_development_dependency(%q<pry-stack_explorer>.freeze, ["~> 0"])
end

18 changes: 17 additions & 1 deletion lib/grade_runner.rb
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
require "grade_runner/runner"
require "grade_runner/utils/path_utils"
require "grade_runner/services/token_service"
require "grade_runner/services/config_service"
require "grade_runner/services/github_service"
require "grade_runner/services/spec_service"
require "grade_runner/services/grade_service"
require "grade_runner/railtie" if defined?(Rails)

module GradeRunner
class Error < StandardError; end

DEFAULT_SUBMISSION_URL = "https://grades.firstdraft.com"

class << self
attr_writer :default_points, :override_local_specs
attr_writer :default_points, :override_local_specs, :submission_url

def default_points
@default_points || 1
Expand All @@ -19,8 +27,16 @@ def override_local_specs
end
end

def submission_url
@submission_url || DEFAULT_SUBMISSION_URL
end

def config
yield self
end

def project_root
Utils::PathUtils.project_root
end
end
end
15 changes: 8 additions & 7 deletions lib/grade_runner/formatters/json_output_formatter.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
RSpec::Support.require_rspec_core "formatters/json_formatter"
require "oj"
require "rspec/core"

class JsonOutputFormatter < RSpec::Core::Formatters::JsonFormatter
RSpec::Core::Formatters.register self, :dump_summary

Expand All @@ -8,7 +9,7 @@ def dump_summary(summary)
examples.
map { |example| example.metadata.fetch(:points, GradeRunner.default_points).to_i }.
sum

earned_points = summary.
examples.
select { |example| example.execution_result.status == :passed }.
Expand All @@ -17,7 +18,7 @@ def dump_summary(summary)

score = (earned_points.to_f / total_points).round(4)
score = 0 if score.nan?

@output_hash[:summary] = {
duration: summary.duration,
example_count: summary.example_count,
Expand All @@ -29,14 +30,14 @@ def dump_summary(summary)
score: score
}
result = (@output_hash[:summary][:score] * 100).round(2)

if summary.errors_outside_of_examples_count.positive?
result = "An error occurred while running tests"
else
result = result.to_s + "%"
end


@output_hash[:summary_line] = [
"#{summary.example_count} #{summary.example_count == 1 ? "test" : "tests"}",
"#{summary.failure_count} failures",
Expand All @@ -48,7 +49,7 @@ def dump_summary(summary)
def close(_notification)
output.write Oj.dump @output_hash
end

private

def format_example(example)
Expand Down
6 changes: 1 addition & 5 deletions lib/grade_runner/railtie.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@

module GradeRunner
class Railtie < ::Rails::Railtie

rake_tasks do
load "tasks/grade_runner.rake"
load "tasks/grade.rake"
end
rake_tasks { load "grade_runner/tasks.rb" }
end
end
2 changes: 0 additions & 2 deletions lib/grade_runner/runner.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

module GradeRunner
class Runner

def initialize(submission_root_url, grades_access_token, rspec_output_json, username, reponame, sha, source)
@submission_url = submission_root_url + submission_path
@grades_access_token = grades_access_token
Expand Down Expand Up @@ -47,6 +46,5 @@ def data
source: @source
}
end

end
end
Loading