Clone
 

noah botimer <noah@botimer.net> in Sakai.Git

SAK-43201 - Improve Component Manager to support testing

This commit reworks the relationship between the ComponentManager cover,

SpringCmpMgr, and SakaiApplicationContext. It doesn't change the way

that normal startup happens, but it does refactor and integrate the code

paths some.

There were lots of challenges because of cyclic dependencies causing

synchronization problems. These are not totally gone, but there is a

path forward. Generally speaking, these happen because beans call the

ComponentManager.get methods during their init. It's pretty easy to foul

it up and get transitive or timing issues. The main purpose of this

change is to start the process of inverting the relationship of the

ComponentManager and the Spring context, in order to normalize how beans

are created and wired up.

Key points:

* There is a new constructor on SpringCmpMgr that allows you to

create an instance that will initialize itself, rather than the cover

doing it and needing to load and refresh outside, later.

* There is a mechanism to register that new instance with the cover

during its startup so the singleton accesses continue to hit the

manager before it is refreshed.

* The IntegrationTestContextLoader uses this new mechanism and gives an

easy extension point for customizing which configs and components

are loaded without resorting to files in special paths.

* The kernel component files are loaded from the classpath, meaning

that they stay in sync with kernel-impl automatically.

* The test configuration is also loaded by classpath and defaults to

reading test.properties from the classpath, meaning the default

in-memory HSQLDB is automatic.

* The application context that launches tests sets the Sakai context as

parent, so all the regular bean config and injection niceties apply

without doing any explicit test harness initialization.

    • -0
    • +27
    /kernel/test-helper/src/main/resources/log4j.properties
SAK-43201 - Add Integration Test Helper

This addition to the kernel is to provide a convenient base for building

integration tests that run against a fully loaded Spring application

context and ComponentManager.

It builds an uberjar to include all of the dependencies needed to start

the kernel under a test suite and introduces the failsafe plugin to

separate unit and integration tests. Adding the test helper as a single

dependency is sufficient to subclass IntegrationTest for an individual

test or, more likely, a base class for a whole tool and then subsequent

individual tests. The failsafe defaults are used, meaning that test

files starting or ending with "IT" will be run with no additional

configuration.

    • -0
    • +165
    /kernel/test-helper/pom.xml
    • -0
    • +26
    /kernel/test-helper/src/assembly/helper-jar.xml
    • -0
    • +12
    /kernel/test-helper/src/main/resources/sakai.test/test.properties
SAK-43092 - Fix up POMs for Eclipse compatibility (#7825)

There were a few modules that would compile with Maven directly, but

throw errors when imported as Eclipse projects. The main two issues were

in the Java EE API and a Jackson coordinate change.

There are two minor code changes that I expect would have broken Maven

builds, but apparently did not. One was a test class/package mismatch

in LTI and the other was some missing method implementations for an XML

class in rwiki.

    • -4
    • +2
    /cloud-storage/googledrive/impl/pom.xml
  1. … 6 more files in changeset.
SAK-43092 - Fix up POMs for Eclipse compatibility (#7825)

There were a few modules that would compile with Maven directly, but

throw errors when imported as Eclipse projects. The main two issues were

in the Java EE API and a Jackson coordinate change.

There are two minor code changes that I expect would have broken Maven

builds, but apparently did not. One was a test class/package mismatch

in LTI and the other was some missing method implementations for an XML

class in rwiki.

(cherry picked from commit a5fd8dfc95d0aa6d75deeda35e063e54c016e852)

    • -4
    • +2
    /cloud-storage/googledrive/impl/pom.xml
  1. … 6 more files in changeset.
SAK-43102 - Extract AnnouncementWrapper (#7828)

This is a first step toward cleaning up AnnouncementAction. It extracts

the AnnouncementWrapper nested class, tidies up the handling of a couple

of entangled constants, removes a few references to TimeService, and

adds a couple of minimal tests to get things moving.

    • -0
    • +15
    /announcement/announcement-tool/tool/pom.xml
SAK-43102 - Extract AnnouncementWrapper (#7828)

This is a first step toward cleaning up AnnouncementAction. It extracts

the AnnouncementWrapper nested class, tidies up the handling of a couple

of entangled constants, removes a few references to TimeService, and

adds a couple of minimal tests to get things moving.

(cherry picked from commit e2dfed627cf769b3691f12740832fd84d2451cff)

    • -0
    • +15
    /announcement/announcement-tool/tool/pom.xml
SAK-43102 - Extract AnnouncementWrapper

This is a first step toward cleaning up AnnouncementAction. It extracts

the AnnouncementWrapper nested class, tidies up the handling of a couple

of entangled constants, removes a few references to TimeService, and

adds a couple of minimal tests to get things moving.

    • -0
    • +15
    /announcement/announcement-tool/tool/pom.xml
SAK-43092 - Fix up POMs for Eclipse compatibility

There were a few modules that would compile with Maven directly, but

throw errors when imported as Eclipse projects. The main two issues were

in the Java EE API and a Jackson coordinate change.

There are two minor code changes that I expect would have broken Maven

builds, but apparently did not. One was a test class/package mismatch

in LTI and the other was some missing method implementations for an XML

class in rwiki.

    • -4
    • +2
    /cloud-storage/googledrive/impl/pom.xml
  1. … 6 more files in changeset.
SAK-43092 - Fix up POMs for Eclipse compatibility

There were a few modules that would compile with Maven directly, but

throw errors when imported as Eclipse projects. The main two issues were

in the Java EE API and a Jackson coordinate change.

There are two minor code changes that I expect would have broken Maven

builds, but apparently did not. One was a test class/package mismatch

in LTI and the other was some missing method implementations for an XML

class in rwiki.

    • -4
    • +2
    /cloud-storage/googledrive/impl/pom.xml
  1. … 5 more files in changeset.
SAK-41442 - Fix ContentItem insertion in Assignments

This is the minimal fix for the problem, but the whole plugin should be

cleaned up. The problem was with multiple editor instances on the page

and some incidental shared state.

The init function registers the ContentItemDialog with the top level

CKEDITOR object for each editor instance. What was happening in this

particular case was that the dialog was overwritten with the binding to

the second instance, for peer assessment, which is hidden. The editor

reference in the onOk callback was not for the actual editor that

invoked the dialog, but the last one initialized.

The error thrown was for the first selection range (range.checkReadOnly)

from insertHtml/insert in editable.js in CKEditor. With the second

editor hidden, the selection range array is empty, so the first item was

undefined. The shared/overwritten reference would have been more obvious

if the second editor was visible -- the item would have been inserted in

the wrong text area.

A more thorough fix would examine the init vs. onLoad details and make

sure that addIframe is called the appropriate number of times and that

only initialization that must run for each editor instance does. I

believe that would be the addCommand and addButton pieces. The

ContentItemIFrameWindow is also global. It is incidentally safe to use

because only one dialog can be in process at a time, but this should

reside within the instance of the dialog itself.

SAM-2725 - Improve "accept until" date handling

This approach addresses multiple consistency issues with "accept until"

dates for assessments. The changes are:

* When saving an assessment with "Yes, accept until" selected, but with

no date entered, set it to "No, not after due date"

* Treat assessments with "Yes, accept until" with no date the same as

asssessments with "No, not after due date" for "active" calculations

* Reflect this slightly different active/inactive status in the

authoring list (authorIndex)

* Reflect the adjusted status on the assessment list for students

(selectIndex)

This also addresses some of SAM-2738 (not considering feedback).

SAM-2725 - Improve "accept until" date handling

This approach addresses multiple consistency issues with "accept until"

dates for assessments. The changes are:

* When saving an assessment with "Yes, accept until" selected, but with

no date entered, set it to "No, not after due date"

* Treat assessments with "Yes, accept until" with no date the same as

asssessments with "No, not after due date" for "active" calculations

* Reflect this slightly different active/inactive status in the

authoring list (authorIndex)

* Reflect the adjusted status on the assessment list for students

(selectIndex)

This also addresses some of SAM-2738 (not considering feedback).

SAM-2724 - Fetch grading attachments eagerly

This is a small change to fetch any grading attachments for just the one

call used in the submission status view. This view does a

BeanUtils.copyProperties from the AssessmentGradingData, so the

uninitialized proxy fails because it is iterated outside the session.

Rather than trying to open the session in the view, this applies a

single fetch.

A more aggressive approach would be to mark the collection

as eager, however, this may cause other places that do not use the

attachments to do lots of additional loading. Here, we know that the

collection is traversed, so fetching is the best answer.

If there is need for optimization, a new DAO signature that explicitly

fetches grading attachments would be appropriate. If there are many

places that need the fetch, the added signature should be to not fetch.

SAM-2724 - Fetch grading attachments eagerly

This is a small change to fetch any grading attachments for just the one

call used in the submission status view. This view does a

BeanUtils.copyProperties from the AssessmentGradingData, so the

uninitialized proxy fails because it is iterated outside the session.

Rather than trying to open the session in the view, this applies a

single fetch.

A more aggressive approach would be to mark the collection

as eager, however, this may cause other places that do not use the

attachments to do lots of additional loading. Here, we know that the

collection is traversed, so fetching is the best answer.

If there is need for optimization, a new DAO signature that explicitly

fetches grading attachments would be appropriate. If there are many

places that need the fetch, the added signature should be to not fetch.

SAM-2717 - Handle copy of null grading attachments

There is a setter for the grading attachment list on

AssessmentGradingData, which wraps an underlying HashSet. When copying,

for example, from AgentResults, where there are no attachments, the

HashSet constructor on the null value throws an

InvocationTargetException, which obscured the problem. This change

creates a new, empty set if null is passed to this setter.

SAM-2717 - Handle copy of null grading attachments

There is a setter for the grading attachment list on

AssessmentGradingData, which wraps an underlying HashSet. When copying,

for example, from AgentResults, where there are no attachments, the

HashSet constructor on the null value throws an

InvocationTargetException, which obscured the problem. This change

creates a new, empty set if null is passed to this setter.

SAK-30325 - Unwrap HTML files for direct link streaming

Because HTML files use a wrapper class that does not extend from

BaseResourceEdit, and the DbContentService has many places that depend

on casting to read the private m_filePath property, a ClassCastException

was being thrown (and swallowed, and turned into a 404 error) when

attempting to view HTML files.

This change is less than ideal, but it works around the problem by

allowing the wrapped resource to be accessed and cast. A better fix

would rework the relationship between these classes (nesting, casting,

type checking) and firm up better interfaces/base classes.

There is also a safety check added before the cast so any other

ContentResource implementations that make it here will just return null

for the direct link -- causing them to be sent by the server directly.

SAK-30325 - Unwrap HTML files for direct link streaming

Because HTML files use a wrapper class that does not extend from

BaseResourceEdit, and the DbContentService has many places that depend

on casting to read the private m_filePath property, a ClassCastException

was being thrown (and swallowed, and turned into a 404 error) when

attempting to view HTML files.

This change is less than ideal, but it works around the problem by

allowing the wrapped resource to be accessed and cast. A better fix

would rework the relationship between these classes (nesting, casting,

type checking) and firm up better interfaces/base classes.

There is also a safety check added before the cast so any other

ContentResource implementations that make it here will just return null

for the direct link -- causing them to be sent by the server directly.

Merge pull request #1737 from ottenhoff/SAK-30347

SAK-30347 close streams after saving to the remote blob store

SAM-2685 - Save audio uploads to content hosting

* Rename media property to dbMedia to separate database and content

hosting handling

By changing the Hibernate mapping for the media column to the dbMedia

property, we can allow getMedia to encapsulate the behavior of using

content hosting or the database. The new dbMedia property is public, so

it can still be used directly (by the DAO). When saving, any media bytes

set are saved to CHS and the dbMedia property is set to null. This will

remove the file from the database row. Subsequent reads will find that

field is null and look for the file in CHS. The previous interface and

behavior are preserved for client code.

* Add batch media conversion

The conversion runs on startup when triggered with

`samigo.convertMedia = true` in sakai.properties (or by another property

mechanism). This uses batching to be safe for restarts or multiple

nodes. Records are processed in batches of 10. If a batch is

interrupted, records may be left with the value of "CONVERTING" in the

location field. To reinitiate these, that field should be set to null

for those records.

SAK-29519 - Fix Swift tests

* Fix package name

* Disable tests by default

* Add swift-tests profile to enable tests

* Add a properties file to configure Swift account

* Update README

    • -0
    • +5
    /cloud-content/impl/src/test/resources/swift.properties
SAK-29519 - Final cleanups per pull request comments

    • -18
    • +0
    /cloud-content/impl/nb-configuration.xml
SAK-29519 - Subtree merge of cloud-content repo

    • -0
    • +1
    /cloud-content/.gitignore
    • -0
    • +71
    /cloud-content/README.md
    • -0
    • +18
    /cloud-content/impl/nb-configuration.xml
    • -0
    • +70
    /cloud-content/impl/pom.xml
    • -0
    • +27
    /cloud-content/pack/pom.xml
    • -0
    • +14
    /cloud-content/pack/src/webapp/WEB-INF/components.xml
    • -0
    • +28
    /cloud-content/pom.xml
  1. … 5 more files in changeset.
SAK-29519 - Remove extraneous diffs

    • -1072
    • +0
    /impl/src/main/diff/Architectual Change to ContentService v2.diff
    • -122
    • +0
    /impl/src/main/diff/KNL-101.diff
    • -143
    • +0
    /impl/src/main/diff/KNL-1283.diff
SAK-29519 - Remove size checks from BlobStore handler

The S3 API does not expose a reasonable way to check account/container

usage, and this is the primary scenario for the BlobStore handler, so

this removes the configuration and size checks.

SAK-29519 - Add base BlobStore/S3 content implementation

* Move everything to a general package name

* Adjust Swift bean definition to use cloudcontent package

* Add BlobStoreFileSystemHandler

* Add .blobstore bean

* Add aws-s3 provider dependency

* Register aws-s3 with the ProviderRegistry (pending a fix for

the @AutoService annotation that is not yet working)

* Update README with S3 instructions

    • -1
    • +5
    /pack/src/webapp/WEB-INF/components.xml
SAK-29519 - Update readme with setup instructions

SAK-29519 - Package up Swift handler for use in Sakai

* Use the typical impl/pack breakdown for service/component

* Work around the ServiceLoader not finding service definitions by

using the ApiRegistry

* Add a top level POM that has master as a parent

    • -0
    • +18
    /impl/nb-configuration.xml
    • -0
    • +1072
    /impl/src/main/diff/Architectual Change to ContentService v2.diff
    • -0
    • +122
    /impl/src/main/diff/KNL-101.diff
    • -0
    • +143
    /impl/src/main/diff/KNL-1283.diff
    • -0
    • +1329
    /impl/src/main/diff/all.diff
    • -0
    • +43
    /impl/src/main/java/coza/opencollab/sakai/swift/Logger.java
    • -0
    • +10
    /pack/src/webapp/WEB-INF/components.xml
    • -1072
    • +0
    /src/main/diff/Architectual Change to ContentService v2.diff
  1. … 7 more files in changeset.
KNL-1268 - Use bean alias to select default content filesystem

SAK-29519 - Upgrade jclouds to 1.9.0 to resolve guava version issues