mirror of
https://github.com/microsoft/HybridRow.git
synced 2025-06-08 01:10:04 +01:00
c36b0adfe8
* Add pom.xml placeholder and Hybrid Row whitepaper * Checkpoint for safe keeping * Progressed on port from dotnet to java * Progressed on port from C# to Java * Brought in Utf8AnyString.java * Progressed on port from C# to Java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Updated documents releated to release of Microsoft Code under MIT License * Updated JRE copyright notices * Progressed on port from dotnet to java * Renamed jre as java because Java is used to can refer to two things: the platform and the language. In this case we are referring to the Java Platform, not the Java Runtime Environment. * Progressed on port * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on dotnet port to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Progressed on port from dotnet to java * Code cleanup, especially in Utf8String class which is much improved. * Progressed on port from dotnet to java. * Simplified Utf8String some more * Code cleanup * Excluding most tests with the expectation that we'll bring them in as we continue to expand our test matrix. * Added and debugged some tests * Added and debugged some tests * Added and debugged some tests * Added, revised, and debugged some tests and test issues * Added, revised, and debugged some tests and test issues * Added, revised, and debugged some tests and test issues * Optimized imports * Removed dead code * Updated java docs * Added, revised, and debugged some tests and test issues * Added, revised, and debugged some tests and test issues * Added, revised, and debugged some tests and test issues * Added, revised, and debugged some tests and test issues. Updated javadocs and pom.xml. We now add SystemSchema.json to azure-cosmos-serialization.jar. Javadocs don't currently build. * Javadoc update * Test update * Progressed on deserializing schemas. * Progressed on deserializing schemas. * Progressed on deserializing schemas. * testLoadSchmea passes * code cleanup in prep for next step: loading a namespace that includes a schema with a udt: the one that the Spark connector team needs. * code cleanup in prep for next step: loading a namespace that includes a schema with a udt: the one that the Spark connector team needs. * RootSegment.json now loads * Progressed on RowReader * Progressed on RowReader * RowReaderTest against RootSegment.json/RootSegment.hybridrow succeeds * RowReaderTest against RootSegment.json/RootSegment.hybridrow succeeds * Move RowScanner and DataItem out of test into main * Move RowScanner and DataItem out of test into main * Move RowScanner and DataItem out of test into main * RowIterable now works. RowIterable and RowScanner now include indexes of array elements. * RowIterable and RowScanner now support additional data types * RowScanner now implements Iterable<DataItem> and RowIterable has been removed. * POM now packages SystemSchema.json and SystemSchema class now loads the packaged SystemSchema.json * Updated pom.xml with required licensing info and attempted to produce javadocs. * Progressed on adding support for unsupported Utf8String features * Refinements to Utf8String and Utf8StringTest * Utf8StringTest improvements * Refinements to Utf8String and Utf8StringTest * Added caching of UTF-16 string caching to Utf8String to reduce the time and space cost of Utf8String.toUtf16(). Also: addressed a number of javadoc issues. * Addressed a number of javadoc issues and the javadocs now build. * Utf8String refinements with inefficient support instead of no support for Utf8String.charAt(int) * Bumped netty.version and updated javadocs for Utf8String * Code cleanup (javadoc improvements + tidying) * Utf8String optimizations * Tweaks * Tweaks
609 lines
13 KiB
JSON
609 lines
13 KiB
JSON
{
|
|
"version": "v1",
|
|
"schemas": [{
|
|
"name": "RootSegment",
|
|
"id": -1,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "PropertyBag",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "RootSegmentPropertyBag",
|
|
"id": 7
|
|
}
|
|
},
|
|
{
|
|
"path": "SegmentInfos",
|
|
"type": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "schema",
|
|
"name": "SegmentInfo",
|
|
"id": 1,
|
|
"nullable": false
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"path": "SnapshotInfos",
|
|
"type": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "schema",
|
|
"name": "SnapshotInfo",
|
|
"id": 5,
|
|
"nullable": false
|
|
}
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "InvalidationManifest",
|
|
"id": -2,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "PropertyBag",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "InvalidationManifestPropertyBag",
|
|
"id": 9
|
|
}
|
|
},
|
|
{
|
|
"path": "StreamInfos",
|
|
"type": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "schema",
|
|
"name": "InvalidationStreamInfo",
|
|
"id": 10,
|
|
"nullable": false
|
|
}
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "SegmentInfo",
|
|
"id": 1,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "LSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "PSID",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "FileId",
|
|
"id": 2
|
|
}
|
|
},
|
|
{
|
|
"path": "SegmentMetadata",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "SegmentMetadata",
|
|
"id": 4
|
|
}
|
|
},
|
|
{
|
|
"path": "ReferenceCount",
|
|
"type": {
|
|
"type": "uint8",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "ZeroUtilizationTimestamp",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "Control",
|
|
"type": {
|
|
"type": "uint8",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "FileId",
|
|
"id": 2,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "StorageAccountIndex",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "FileType",
|
|
"type": {
|
|
"type": "int16",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "FileIndex",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "PartitionId",
|
|
"type": {
|
|
"type": "guid",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "PartitionKeyRangeRid",
|
|
"type": {
|
|
"type": "binary",
|
|
"storage": "fixed",
|
|
"length": 30,
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "SequenceVector",
|
|
"id": 3,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "GlobalSequenceNumber",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "LocalSequenceNumber",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "SegmentMetadata",
|
|
"id": 4,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "InitialRecordCount",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "TombstoneRecordCount",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "InvalidatedRecordCount",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "SegmentSize",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "MinSequenceVector",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "SequenceVector",
|
|
"id": 3
|
|
}
|
|
},
|
|
{
|
|
"path": "MaxSequenceVector",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "SequenceVector",
|
|
"id": 3
|
|
}
|
|
},
|
|
{
|
|
"path": "InvalidationFileId",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "FileId",
|
|
"id": 2
|
|
}
|
|
},
|
|
{
|
|
"path": "InvalidationStats",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "InvalidationStats",
|
|
"id": 6
|
|
}
|
|
},
|
|
{
|
|
"path": "SchemaHash",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "ExpiredRecordCount",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "MinRecordTimestamp",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "MaxRecordTimestamp",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "PartialSegment",
|
|
"type": {
|
|
"type": "bool",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "SnapshotInfo",
|
|
"id": 5,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "SnapshotId",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "Timestamp",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "PSID",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "FileId",
|
|
"id": 2
|
|
}
|
|
},
|
|
{
|
|
"path": "TTL",
|
|
"type": {
|
|
"type": "int32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "InvalidationStats",
|
|
"id": 6,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "Count",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "SizeInBytes",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "RootSegmentPropertyBag",
|
|
"id": 7,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "Control",
|
|
"type": {
|
|
"type": "uint8",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "NumberOfEntries",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "NumberOfSnapshots",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "LastCheckpointedLSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "LastCheckpointedSequenceVector",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "SequenceVector",
|
|
"id": 3
|
|
}
|
|
},
|
|
{
|
|
"path": "LastFlushedLSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "LastFlushedSequenceVector",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "SequenceVector",
|
|
"id": 3
|
|
}
|
|
},
|
|
{
|
|
"path": "LastRestoredSourceOffset",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "Logoffset",
|
|
"id": 8
|
|
}
|
|
},
|
|
{
|
|
"path": "LogVersion",
|
|
"type": {
|
|
"type": "uint8",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "Timestamp",
|
|
"type": {
|
|
"type": "uint64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "ArchivalPartitionId",
|
|
"type": {
|
|
"type": "guid",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "TTL",
|
|
"type": {
|
|
"type": "int32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "PhysicalUsageInKB",
|
|
"type": {
|
|
"type": "uint64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "LogicalUsageInKB",
|
|
"type": {
|
|
"type": "uint64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "Logoffset",
|
|
"id": 8,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "LSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "RecordGroupID",
|
|
"type": {
|
|
"type": "int32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "RecordIndex",
|
|
"type": {
|
|
"type": "int32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "InvalidationManifestPropertyBag",
|
|
"id": 9,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "NumberOfEntries",
|
|
"type": {
|
|
"type": "uint32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "MaxMergedLevel0LSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "Timestamp",
|
|
"type": {
|
|
"type": "uint64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"name": "InvalidationStreamInfo",
|
|
"id": 10,
|
|
"type": "schema",
|
|
"properties": [{
|
|
"path": "InvalidationFileId",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "FileId",
|
|
"id": 2
|
|
}
|
|
},
|
|
{
|
|
"path": "State",
|
|
"type": {
|
|
"type": "uint8",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "StartLSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "EndLSID",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "Level",
|
|
"type": {
|
|
"type": "int32",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "CreationTimestamp",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "ObsoletionTimestamp",
|
|
"type": {
|
|
"type": "int64",
|
|
"storage": "fixed",
|
|
"nullable": false
|
|
}
|
|
},
|
|
{
|
|
"path": "InvalidationStats",
|
|
"type": {
|
|
"type": "schema",
|
|
"name": "InvalidationStats",
|
|
"id": 6
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
} |