araxiaonline/WoWDBDefs
3
0
Fork 0
mirror of https://github.com/araxiaonline/WoWDBDefs.git synced 2026-06-13 03:32:22 -04:00
Code Issues Packages Projects Releases Wiki Activity

trailing whitespace

Browse Source
This commit is contained in:
Ferronn
2022-06-24 16:58:19 +00:00
parent e1951e9e8f
commit b8f5b962c5
25 changed files with 63 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.pre-commit-config.yaml
View File

@@ -6,4 +6,4 @@ repos:
- id: check-json
- id: check-yaml
- id: end-of-file-fixer
# - id: trailing-whitespace TODO
- id: trailing-whitespace

14
UPDATING.md
View File

@@ -11,10 +11,10 @@ The following is a short step-by-step tutorial on how to update DBDs in a best-c
## Updating DBDs with newer builds (long version)
### Dumping DBMeta to DBD
#### What is DBMeta?
Metadata for WoW's database tables (DB2s in modern WoW) containing field types, sizes, flags and more are contained in the WoW executable and are commonly referred to as "DBMeta" or "DB2Meta".
Metadata for WoW's database tables (DB2s in modern WoW) containing field types, sizes, flags and more are contained in the WoW executable and are commonly referred to as "DBMeta" or "DB2Meta".
#### Generating DBD files from DBMeta
The DBDefsDumper project (which can be found in the `code/C#/DBDefsDumper` directory) reads these metadata sections from the executable and generates a human readable DBD formatted file for each DB2 available/used in the client.
The DBDefsDumper project (which can be found in the `code/C#/DBDefsDumper` directory) reads these metadata sections from the executable and generates a human readable DBD formatted file for each DB2 available/used in the client.
1. Compile the project in `code/C#/DBDefsDumper`.
2. Grab the WoW executable you want to generate definitions out of.
@@ -27,7 +27,7 @@ If dumping goes according to plan and the patterns work, you can skip the rest o
DBMeta entries are located based on patterns that each DB2Meta entry conforms to. These patterns are defined in PatternBuilder.cs for a specific set of builds/patches. Patterns for older builds might be inaccurate/missing, this is something that needs work so we can support wider ranges of builds. However, most importantly, the currently used pattern at time of writing has been in use since 8.0.1, but this may break at any time, likely during the start of a new expansion/patch testing phase. When it does, someone knowledgable will need to reverse engineer the changes to the DB2Meta structure and create a new pattern for that specific patch (and forward). Historically they add new fields every now and then or shift stuff around a bit, so one might be able to get away by simply diffing the structures for the same DBMeta for a single DB2 between two builds and seeing if anything got obviously shifted around. If the changes are more complex, it will likely need proper analysis/reversing.
### Merging "raw" DBD dumps with existing DBDs
After raw definitions are dumped from DBMeta, it's time to merge these up with existing DBDs. Raw definitions have no names and are only for a single build, while existing DBDs have all kinds of builds and, hopefully, contain named fields.
After raw definitions are dumped from DBMeta, it's time to merge these up with existing DBDs. Raw definitions have no names and are only for a single build, while existing DBDs have all kinds of builds and, hopefully, contain named fields.
To merge raw DBDs with existing DBDs:
1. Compile the project in `code/C#/DBDefsMerge`.
@@ -36,16 +36,16 @@ To merge raw DBDs with existing DBDs:
Merging is primarily done based on layouthash (when available), if not available it'll try builds, if those aren't available it'll try build ranges. The merger -should- work fine when ran between raw DBDs and the DBDs from this repo, but things might get funky when working with older builds/raw definitions, especially if layouthashes aren't available.
### Validating definitions
There is a DBDefsValidator project to check the validity of all DBDs format-wise as well as to check if referenced foreign key definitions are valid.
There is a DBDefsValidator project to check the validity of all DBDs format-wise as well as to check if referenced foreign key definitions are valid.
To run it:
1. Compile the project in `code/C#/DBDefsValidator`.
2. Run `dotnet DBDefsValidator.dll <definitionsdir> (rewrite when done: bool, default false) (verbose: bool, default true) (rawRepoDir: location of WoWDBDefsRaw repository, default none)`
It will output whether or not each definition is valid.
It will output whether or not each definition is valid.
Optionally, if ran with `true` as second argument it will rewrite definitions (getting rid of excess whitespace/lines as well reordering definitions properly). This is highly recommended after manually editing definitions.
Setting the third (also optional) argument to `true` will output more verbose output.
The final optional argument is experimental, but can be the raw definition directory to check for any copy-paste errors (e.g. bitsize being wrong).
Setting the third (also optional) argument to `true` will output more verbose output.
The final optional argument is experimental, but can be the raw definition directory to check for any copy-paste errors (e.g. bitsize being wrong).
### Defining unknown fields
Unknown fields are named `Field_x_x_x_xxxxx_yyy` where x is the build and y is the field index. Usually the changes are minor with a field either being added, resized or removed. Sometimes Blizzard "reshuffles" many fields at the same time which is more annoying to map. Sometimes it's pretty easy to tell what changed based on the field types/sizes, sometimes you might need to look at the content of fields through one of the available tools to see what went where. For entirely new fields, it might be useful looking at the structures defined in Blizzard's Lua files or other DB2s that changed in the same build or features that Blizzard is currently working on. After defining fields manually, I suggest running the validator with rewrites on to make sure no orphaned column definitions or useless whitespace stays around.

2
code/C#/DBDefsConverter/DBDXMLSerializer.cs
View File

@@ -19,7 +19,7 @@ namespace DBDefsConverter
public DBDXMLSerializer()
{
// create serializer
// create serializer
_serializer = new XmlSerializer(typeof(SerializableDBDefinition), CreateOverrides());
// build the fieldinfo lookup for ColumnDefinition

2
code/C#/DBDefsCoverage/Program.cs
View File

@@ -77,7 +77,7 @@ namespace DBDefsCoverage
csv.NextRecord();
}
csv.Flush();
csv.Flush();
}
}
}

8
code/C#/DBDefsDumper/DBMeta.cs
View File

@@ -41,11 +41,11 @@ namespace DBDefsDumper
public long dbFilenameOffs;
public int siblingTableHash;
public long namesInFileOffs;
/*
/*
//probs not in osx
const char** field_names_in_file;
const char** field_names;
const char* fk_clause;
const char** field_names_in_file;
const char** field_names;
const char* fk_clause;
*/
}
}

2
code/C#/DBDefsDumper/EXEParsing.cs
View File

@@ -71,7 +71,7 @@ namespace DBDefsDumper
if (command.id == 25)
{
var segmentOffset = offset + 4 + 4 + 16; // Segment start + id + size + name;
var segmentOffset = offset + 4 + 4 + 16; // Segment start + id + size + name;
var vmemOffs = stream.ReadUInt64(segmentOffset);
var vmemSize = stream.ReadUInt64(segmentOffset + 8);
var fileOffs = stream.ReadUInt64(segmentOffset + 16);

16
code/C#/DBDefsDumper/Program.cs
View File

@@ -82,7 +82,7 @@ namespace DBDefsDumper
// Retry with backup pattern (crash log output)
bin.BaseStream.Position = 0;
buildPattern = new byte?[] { 0x00, 0x3C, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6F, 0x6E, 0x3E, 0x20 }; // <Version>
buildPattern = new byte?[] { 0x00, 0x3C, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6F, 0x6E, 0x3E, 0x20 }; // <Version>
buildPatternLength = buildPattern.Length;
while (true)
@@ -128,7 +128,7 @@ namespace DBDefsDumper
// Retry with RenderService pattern..
bin.BaseStream.Position = 0;
buildPattern = new byte?[] { 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x20, null, null, null, null, null, 0x00 }; // <Version>
buildPattern = new byte?[] { 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x20, null, null, null, null, null, 0x00 }; // <Version>
buildPatternLength = buildPattern.Length;
while (true)
@@ -379,7 +379,7 @@ namespace DBDefsDumper
var name = bin.ReadCString();
metas.TryAdd(Path.GetFileNameWithoutExtension(name), meta);
}
bin.BaseStream.Position = matchPos + patternLength;
}
else
@@ -443,7 +443,7 @@ namespace DBDefsDumper
var field_types_in_file = ReadFieldArray(bin, fieldCount, (long)translate((ulong)meta.Value.field_types_in_file_offs));
var field_flags_in_file = ReadFieldArray(bin, fieldCount, (long)translate((ulong)meta.Value.field_flags_in_file_offs));
var field_names_in_file = ReadFieldOffsetArray(bin, fieldCount, (long)translate((ulong)meta.Value.namesInFileOffs));
if (meta.Value.id_column == -1)
{
writer.WriteLine("int ID");
@@ -601,7 +601,7 @@ namespace DBDefsDumper
if (build.StartsWith("6."))
{
var supposedSize = 0;
if((typeFlags.Item1 == "uint" || typeFlags.Item1 == "int") && typeFlags.Item2 != 32)
{
supposedSize = typeFlags.Item2 / 8;
@@ -987,11 +987,11 @@ namespace DBDefsDumper
return patternList;
}
}
#region BinaryReaderExtensions
static class BinaryReaderExtensios
{
/// <summary> Reads the NULL terminated string from
/// <summary> Reads the NULL terminated string from
/// the current stream and advances the current position of the stream by string length + 1.
/// <seealso cref="BinaryReader.ReadString"/>
/// </summary>
@@ -1000,7 +1000,7 @@ namespace DBDefsDumper
return reader.ReadCString(Encoding.UTF8);
}
/// <summary> Reads the NULL terminated string from
/// <summary> Reads the NULL terminated string from
/// the current stream and advances the current position of the stream by string length + 1.
/// <seealso cref="BinaryReader.ReadString"/>
/// </summary>

2
code/C#/DBDefsLib/DBDReader.cs
View File

@@ -34,7 +34,7 @@ namespace DBDefsLib
var columnDefinition = new ColumnDefinition();
/* TYPE READING */
// List of valid types, uint should be removed soon-ish
// List of valid types, uint should be removed soon-ish
var validTypes = new List<string> { "uint", "int", "float", "string", "locstring" };
// Check if line has a space in case someone didn't assign a type to a column name

4
code/C#/DBDefsMerge/Program.cs
View File

@@ -18,7 +18,7 @@ namespace DBDefsMerge
}
var numLayoutsAdded = 0;
var firstDir = args[0];
var secondDir = args[1];
var targetDir = args[2];
@@ -415,7 +415,7 @@ namespace DBDefsMerge
}
}
// Run through column definitions to see if there's any unused columns
// Run through column definitions to see if there's any unused columns
var columnDefinitionsCopy = definitionCopy.columnDefinitions.ToList();
foreach (var columnDefinition in columnDefinitionsCopy)
{

2
code/C#/DBDefsTest/Program.cs
View File

@@ -14,7 +14,7 @@ namespace DBDTest
public static Dictionary<string, DBDefinition> definitionCache = new Dictionary<string, DBDefinition>();
public static Dictionary<string, List<string>> duplicateFileLookup = new Dictionary<string, List<string>>();
public static bool foundError = false;
private static string dbcDir;
static void Main(string[] args)

4
exedumper/initial_dump_from_binary-3.0.1.8303.ps1
View File

@@ -1,4 +1,4 @@
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
{
"Run as Administrator, or inject will fail"
exit
@@ -6,7 +6,7 @@ if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]:
$ErrorActionPreference = "Stop"
$PSDefaultParameterValues['*:Encoding'] = 'utf8'
$script_dir = (split-path $MyInvocation.MyCommand.Path)
$script_dir = (split-path $MyInvocation.MyCommand.Path)
$msvcpp = "Visual Studio 15 2017"
$msvs_short = "vs2017"

9
exedumper/initial_dump_from_binary-3.0.1.8303/dump.cpp
View File

@@ -25,7 +25,7 @@ fun<void (int, const char *, const char **, _UNKNOWN *, char, _UNKNOWN *, _UNKNO
fun<int()> sub_5B1AD0 = 0x5B1AD0;
fun<int()> sub_405AA0 = 0x405AA0;
void on_inject()
{
//! This function is _not_ dbmeta but db update registration. the fields in here are in memory, not in file!
@@ -61,7 +61,7 @@ void on_inject()
}
of << "\n";
of << "BUILD 3.0.1.8303\n";
for (int f = 0; f < fieldCount; ++f) {
std::string suff;
switch(fieldTypesIsh[f]) {
@@ -92,7 +92,7 @@ void on_inject()
}
, false
);
hook (sub_405AA0, [] { sub_5B1AD0(); exit (0); return 0; });
}
@@ -100,7 +100,6 @@ void on_inject()
extern "C" void __declspec(dllexport) __stdcall NativeInjectionEntryPoint(REMOTE_ENTRY_INFO* inRemoteInfo)
{
on_inject();
RhWakeUpProcess();
}

4
exedumper/initial_dump_from_binary-3.0.1.8303/injector.cpp
View File

@@ -21,7 +21,7 @@ int wmain(int argc, WCHAR* argv[])
exe = exe.substr (0, exe.find (' '));
std::wstring command_line (argv[2]);
command_line = command_line.substr (command_line.find (' ') + 1);
ULONG pid;
NTSTATUS nt = RhCreateAndInject
( const_cast<WCHAR*> (exe.c_str())
@@ -40,6 +40,6 @@ int wmain(int argc, WCHAR* argv[])
std::wcout << "RhCreateAndInject failed with error code = " << nt << "\n " << RtlGetLastErrorString() << "\n";
return 1;
}
return 0;
}

2
exedumper/initial_dump_from_binary-3.0.1.8303/patching.hpp
View File

@@ -2,7 +2,7 @@
template<typename T, bool unprotect>
struct var;
template<typename Fun>
template<typename Fun>
struct fun;
template<typename Fun>

6
exedumper/initial_dump_from_binary-3.0.1.8303/patching.ipp
View File

@@ -1,5 +1,5 @@
namespace
namespace
{
template<bool, typename> struct maybe_unprotect;
template<typename T> struct maybe_unprotect<true, T>
@@ -25,7 +25,7 @@ namespace
};
}
namespace
namespace
{
char* module_base()
{
@@ -49,7 +49,7 @@ template<typename T, bool unprotect = true>
struct var
{
size_t const _offset;
constexpr var (size_t offset) : _offset (offset) {}
T* _x = nullptr;

4
exedumper/initial_dump_from_binary-3.0.2.8885.ps1
View File

@@ -1,4 +1,4 @@
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
{
"Run as Administrator, or inject will fail"
exit
@@ -6,7 +6,7 @@ if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]:
$ErrorActionPreference = "Stop"
$PSDefaultParameterValues['*:Encoding'] = 'utf8'
$script_dir = (split-path $MyInvocation.MyCommand.Path)
$script_dir = (split-path $MyInvocation.MyCommand.Path)
$msvcpp = "Visual Studio 15 2017"
$msvs_short = "vs2017"

9
exedumper/initial_dump_from_binary-3.0.2.8885/dump.cpp
View File

@@ -25,7 +25,7 @@ fun<void (int, const char *, const char **, _UNKNOWN *, char, _UNKNOWN *, _UNKNO
fun<int()> sub_5B1AD0 = 0x5D9C70;
fun<void()> sub_405AA0 = 0x405F10;
void on_inject()
{
//! This function is _not_ dbmeta but db update registration. the fields in here are in memory, not in file!
@@ -61,7 +61,7 @@ void on_inject()
}
of << "\n";
of << "BUILD 3.0.2.8885\n";
for (int f = 0; f < fieldCount; ++f) {
std::string suff;
switch(fieldTypesIsh[f]) {
@@ -92,7 +92,7 @@ void on_inject()
}
, false
);
hook (sub_405AA0, [] { sub_5B1AD0(); exit (0); });
}
@@ -100,7 +100,6 @@ void on_inject()
extern "C" void __declspec(dllexport) __stdcall NativeInjectionEntryPoint(REMOTE_ENTRY_INFO* inRemoteInfo)
{
on_inject();
RhWakeUpProcess();
}

4
exedumper/initial_dump_from_binary-3.0.2.8885/injector.cpp
View File

@@ -21,7 +21,7 @@ int wmain(int argc, WCHAR* argv[])
exe = exe.substr (0, exe.find (' '));
std::wstring command_line (argv[2]);
command_line = command_line.substr (command_line.find (' ') + 1);
ULONG pid;
NTSTATUS nt = RhCreateAndInject
( const_cast<WCHAR*> (exe.c_str())
@@ -40,6 +40,6 @@ int wmain(int argc, WCHAR* argv[])
std::wcout << "RhCreateAndInject failed with error code = " << nt << "\n " << RtlGetLastErrorString() << "\n";
return 1;
}
return 0;
}

2
exedumper/initial_dump_from_binary-3.0.2.8885/patching.hpp
View File

@@ -2,7 +2,7 @@
template<typename T, bool unprotect>
struct var;
template<typename Fun>
template<typename Fun>
struct fun;
template<typename Fun>

6
exedumper/initial_dump_from_binary-3.0.2.8885/patching.ipp
View File

@@ -1,5 +1,5 @@
namespace
namespace
{
template<bool, typename> struct maybe_unprotect;
template<typename T> struct maybe_unprotect<true, T>
@@ -25,7 +25,7 @@ namespace
};
}
namespace
namespace
{
char* module_base()
{
@@ -49,7 +49,7 @@ template<typename T, bool unprotect = true>
struct var
{
size_t const _offset;
constexpr var (size_t offset) : _offset (offset) {}
T* _x = nullptr;

4
exedumper/initial_dump_from_binary-3.0.8.9328.ps1
View File

@@ -1,4 +1,4 @@
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
{
"Run as Administrator, or inject will fail"
exit
@@ -6,7 +6,7 @@ if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]:
$ErrorActionPreference = "Stop"
$PSDefaultParameterValues['*:Encoding'] = 'utf8'
$script_dir = (split-path $MyInvocation.MyCommand.Path)
$script_dir = (split-path $MyInvocation.MyCommand.Path)
$msvcpp = "Visual Studio 15 2017"
$msvs_short = "vs2017"

9
exedumper/initial_dump_from_binary-3.0.8.9328/dump.cpp
View File

@@ -25,7 +25,7 @@ fun<void (int, const char *, const char **, _UNKNOWN *, char, _UNKNOWN *, _UNKNO
fun<int()> sub_5B1AD0 = 0x5DF1D0;
fun<void()> sub_405AA0 = 0x406000;
void on_inject()
{
//! This function is _not_ dbmeta but db update registration. the fields in here are in memory, not in file!
@@ -61,7 +61,7 @@ void on_inject()
}
of << "\n";
of << "BUILD 3.0.8.9328\n";
for (int f = 0; f < fieldCount; ++f) {
std::string suff;
switch(fieldTypesIsh[f]) {
@@ -92,7 +92,7 @@ void on_inject()
}
, false
);
hook (sub_405AA0, [] { sub_5B1AD0(); exit (0); });
}
@@ -100,7 +100,6 @@ void on_inject()
extern "C" void __declspec(dllexport) __stdcall NativeInjectionEntryPoint(REMOTE_ENTRY_INFO* inRemoteInfo)
{
on_inject();
RhWakeUpProcess();
}

4
exedumper/initial_dump_from_binary-3.0.8.9328/injector.cpp
View File

@@ -21,7 +21,7 @@ int wmain(int argc, WCHAR* argv[])
exe = exe.substr (0, exe.find (' '));
std::wstring command_line (argv[2]);
command_line = command_line.substr (command_line.find (' ') + 1);
ULONG pid;
NTSTATUS nt = RhCreateAndInject
( const_cast<WCHAR*> (exe.c_str())
@@ -40,6 +40,6 @@ int wmain(int argc, WCHAR* argv[])
std::wcout << "RhCreateAndInject failed with error code = " << nt << "\n " << RtlGetLastErrorString() << "\n";
return 1;
}
return 0;
}

2
exedumper/initial_dump_from_binary-3.0.8.9328/patching.hpp
View File

@@ -2,7 +2,7 @@
template<typename T, bool unprotect>
struct var;
template<typename Fun>
template<typename Fun>
struct fun;
template<typename Fun>

6
exedumper/initial_dump_from_binary-3.0.8.9328/patching.ipp
View File

@@ -1,5 +1,5 @@
namespace
namespace
{
template<bool, typename> struct maybe_unprotect;
template<typename T> struct maybe_unprotect<true, T>
@@ -25,7 +25,7 @@ namespace
};
}
namespace
namespace
{
char* module_base()
{
@@ -49,7 +49,7 @@ template<typename T, bool unprotect = true>
struct var
{
size_t const _offset;
constexpr var (size_t offset) : _offset (offset) {}
T* _x = nullptr;