Add new shape descriptor controls, refactor various systems to handle the new possibilities

Author: SteelFill

Source/Documentation/Manual/features-rollingstock.rst

@@ -411,6 +411,121 @@ some steps of the content creation process or allow more control over content th
 possible. The goal of these features is to save content creators' time, give additional power to
 creators, and to simplify the installation process for end users.
 
+Runtime shape manipulation
+--------------------------
+
+.. _features-shape-manipulation:
+
+MSTS shape files can be clunky to work with due to the proprietary nature of the file format
+and the limitations of the tools available to manipulate them. Even when a shape has been
+manipulated, it's often not appropriate to redistribute those changes (unless the end user
+is to replicate the manipulation themselves) to avoid plagarism. To improve this, OR now
+supports additional ways to manipulate shape file data in memory without editing the original
+shape.
+
+These features use the shape descriptor (.sd) file to provide the information needed to change
+the shape file data at runtime. The .sd file can be edited in plain text in standard text editors,
+and there is no risk of plagarism when including .sd files in a download, so changes can be
+made and distributed easily. However, it may be desired to use a different .sd file than the
+default one (the .sd default file has the same name as the .s file) out of a desire to not overwrite
+the original, or to re-use the same shape file repeatedly with different data, saving file space.
+To facilitate this, most places in wagons and engines where a .s file can be specified now accept 
+an additional input to *specify a different .sd file than the default*.
+
+For example, ``WagonShape ( "dash9.s" "dash9_reskin.sd" )`` would load the dash9 shape, but
+instead of loading dash9.sd, would load dash9_reskin.sd and apply any settings in that alternate .sd file.
+Similar can be applied to ORTS freight animations (not MSTS freight animations), passenger views,
+3D cabs, and animated couplers. Note that both the .s and .sd file specified can be
+in a different folder from the .eng or .wag by using relative file paths. If only the .s file is
+given, OR will assume the .sd file has the same name as the given .s file, just like MSTS. OR does
+not require .sd files to function, so if the .sd file is missing OR will continue with default settings.
+
+While .sd file replacement is currently unique to engines and wagons, the new features inside .sd files
+can be applied to any .sd file, including scenery.
+
+.. index::
+   single: ESD_ORTSTextureReplacement
+
+Texture Replacement
+^^^^^^^^^^^^^^^^^^^
+
+Perhaps one of the most useful additional shape descriptor features is the ability to replace the
+textures used by a shape without editing, copying, or even moving the shape file. To achieve this,
+add the ``ESD_ORTSTextureReplacement`` parameter to the .sd file, and enter texture names in the
+format ``ESD_ORTSTextureReplacement ( OriginalTexture.ace ReplacementTexture.ace )``. Any
+OriginalTexture not specified won't be changed, and if the shape doesn't have a texture specified,
+then a warning message is produced. This works with both .ace and .dds textures. Multiple
+textures can be replaced in one go by adding additional pairs of textures::
+
+    SIMISA@@@@@@@@@@JINX0t1t______
+
+    shape ( BNSF_C44_9W_4614.s
+	    ESD_Detail_Level ( 0 )
+	    ESD_Alternative_Texture ( 0 )
+	    ESD_Bounding_Box ( -1.632 -0.095 -11.05 1.684 4.628 11.05 )
+        ESD_ORTSTextureReplacement (
+            "BNSF_C449W_4410a.dds"    "BNSF_C449W_4614a.dds"
+            "BNSF_C449W_4410b.dds"    "BNSF_C449W_4614b.dds"
+        )
+    )
+
+This would reskin BNSF 4410 into BNSF 4614 without any need to edit the original shape. Remember
+to reference the original .s file, but with a custom .sd file ``WagonShape ( "..\\BNSF_MULLAN_GE_ENGINES\\BNSF_C44_9W_4410.s" 
+"BNSF_C44_9W_4614.sd" )`` so that the original shape file doesn't need to be copied. It is recommended
+that content creators making multiple skins of a new model, or reskinning an existing model, use this
+method to provide different textures for different engines/wagons as only one copy of the shape file is
+needed, reducing install size and simplifying installation as users don't need to move/copy/edit any
+shape files.
+
+Translation Matrix Modification
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+All shape files are organized into one or more *sub objects*, where each sub object is positioned/scaled/rotated
+relative to the other sub objects using a *transformation matrix*, which are the named parts of the
+shape that can be seen in shape viewing utilities. A consequence of this is that the
+position/scale/rotation of a sub object can be changed without changing any of the 3D data used
+to draw the sub object. This can be useful to correct errors in the position/scale/rotation
+of the whole, or part of, a model without changing a tremendous amount of data.
+
+.. index::
+   single: ESD_ORTSMatrixTranslation
+   single: ESD_ORTSMatrixScale
+   single: ESD_ORTSMatrixRotation
+
+- To change the position of a sub object, use ``ESD_ORTSMatrixTranslation ( MATRIX x y z )`` where
+  ``MATRIX`` is the name of the matrix and ``x y z`` are respectively the +right/-left, +up/-down,
+  and +front/-back offsets from the original position, measured in units of distance (meters by default).
+- To change the scale (size) of a sub object, use ``ESD_ORTSMatrixScale ( MATRIX x y z )`` where
+  ``MATRIX`` is the name of the matrix and ``x y z`` are the horizontal, vertical, and lengthwise
+  scale factors from the original scale. Scale factors larger than 1 increase size in that dimension,
+  between 0 and 1 reduce size, and negative scale factor mirrors the object in that dimension.
+- To change the rotation of a sub object, use ``ESD_ORTSMatrixRotation ( MATRIX y p r )`` where
+  ``MATRIX`` is the name of the matrix and ``y p r`` are the yaw (+left/-right), pitch (+up/-down)
+  and roll (+ccw/-cw) angle change from the original rotation, measured in radians by default. The ``deg``
+  unit suffix can be used to give angles in degrees.
+
+Only one of each type of parameter can be provided for each matrix. If all 3 transformations are
+applied to a matrix, they will be applied in the order of *translation, scale, then rotation*.
+An important consideration when manipulating these properties of transformation matrices is that
+the transformation will also be applied to any sub objects *below* the current matrix in the hierarchy.
+The transformation also affects 3D interiors, particle emitters, lights, and sounds attached to any
+affected sub objects.
+Should the transformation be desired only for something higher in the hierarchy, an equal but opposite
+transformation would be required on the lower-level sub object. It should also be considered that
+changes made to matrices are not purely graphical and could have consequences with some simulation
+systems. Extreme settings may break simulation behavior.
+
+.. index::
+   single: ESD_ORTSMatrixRename
+
+The name of the transformation matrix is itself important for simulator behaviors, particularly
+animations and determining the structure of rolling stock. Should a matrix be named incorrectly,
+or a change in behavior be desired, a matrix can be renamed using ``ESD_ORTSMatrixReanme ( OLDNAME NEWNAME )``
+in the .sd file. OR will scan for any matrix "OLDNAME" and change the name to "NEWNAME". If no
+matrix with the old name can be found, a warning will be produced and nothing will change. Note
+that the matrix rename step occurs *after* the previously described matrix modifications, so
+the old matrix name must be used by any other .sd parameters that require a matrix name.
+
 Automatic wagon size calculation
 --------------------------------
 
@@ -427,7 +542,9 @@ To simplify this process, and produce more reasonable dimensions for rolling sto
 automatically calculate the dimensions of rolling stock based on the shape file used. Enter
 ``ORTSAutoSize`` in the Wagon section of an engine or wagon to allow OR to determine
 the width, height, and length of the rolling stock based on the dimensions of the main shape file,
-ignoring any values entered manually in the MSTS Size parameter.
+ignoring any values entered manually in the MSTS Size parameter. Note that this process is
+not aware of any :ref:`shape descriptor overrides <features-shape-manipulation>` so any changes made
+to the shape there will not be reflected in the automatically calculated size.
 
 ``ORTSAutoSize`` accepts 3 (optional) arguments, default units in meters, corresponding to offsets from the
 shape's width, height, and length respectively. For example, ``ORTSAutoSize ( 0.1m, -0.2m, -0.18m )``
@@ -1157,6 +1274,7 @@ Here below a sample of a ``.load-or`` file::
   	{
 	  	"Name" : "triton",
 	  	"Shape" : "COMMON_Container_3d\\Cont_40ftHC\\container-40ftHC_Triton.s",
+	  	"ShapeDescriptor" : "COMMON_Container_3d\\Cont_40ftHC\\container-40ftHC_Triton.sd",
 	  	"ContainerType" : "C40ftHC",
 	  	"IntrinsicShapeOffset": [0,1.175,0],
    		"EmptyMassKG": 2100.,
@@ -1167,7 +1285,10 @@ Here below a sample of a ``.load-or`` file::
 - "Container" is a fixed keyword.
 - "Name" has as value a string used by Open Rails when the container must be indentified in a message 
   to the player.
-- "Shape" has as value the path of the container shape, having ``Trainset`` as base.
+- "Shape" has as value the path of the container shape (.s) file, having ``Trainset`` as base.
+- "ShapeDescriptor" has the path of the container shape descriptor (.sd) file,
+  having ``Trainset`` as base. This is optional; if missing OR assumes the shape
+  descriptor is in the same location with the same name as the shape file.
 - "ContainerType" identifies the container type, which may be one of the following ones::
 
   * C20ft

Source/Orts.Formats.Msts/ShapeDescriptorFile.cs

@@ -17,6 +17,7 @@
 
 using System;
 using System.Collections.Generic;
+using Microsoft.Xna.Framework;
 using Orts.Parsers.Msts;
 
 namespace Orts.Formats.Msts
@@ -69,7 +70,7 @@ public SDShape()
 
             public SDShape(STFReader stf)
             {
-                stf.ReadString(); // Ignore the filename string. TODO: Check if it agrees with the SD file name? Is this important?
+                stf.ReadString();
                 stf.ParseBlock(new STFReader.TokenProcessor[] {
                     new STFReader.TokenProcessor("esd_detail_level", ()=>{ ESD_Detail_Level = stf.ReadIntBlock(null); }),
                     new STFReader.TokenProcessor("esd_alternative_texture", ()=>{ ESD_Alternative_Texture = stf.ReadIntBlock(null); }),
@@ -84,9 +85,22 @@ public SDShape(STFReader stf)
                     new STFReader.TokenProcessor("esd_ortssoundfilename", ()=>{ ESD_SoundFileName = stf.ReadStringBlock(null); }),
                     new STFReader.TokenProcessor("esd_ortsbellanimationfps", ()=>{ ESD_CustomAnimationFPS = stf.ReadFloatBlock(STFReader.UNITS.Frequency, null); }),
                     new STFReader.TokenProcessor("esd_ortscustomanimationfps", ()=>{ ESD_CustomAnimationFPS = stf.ReadFloatBlock(STFReader.UNITS.Frequency, null); }),
+                    new STFReader.TokenProcessor("esd_ortstexturereplacement", ()=>{ ParseReplacementStrings(stf, ref ESD_TextureReplacement); }),
+                    new STFReader.TokenProcessor("esd_ortsmatrixrename", ()=>{ ParseReplacementStrings(stf, ref ESD_MatrixRename); }),
+                    new STFReader.TokenProcessor("esd_ortsmatrixtranslation", ()=>{ ParseMatrixOverride(STFReader.UNITS.Distance, stf, ref ESD_MatrixTranslation); }),
+                    new STFReader.TokenProcessor("esd_ortsmatrixscale", ()=>{ ParseMatrixOverride(STFReader.UNITS.None, stf, ref ESD_MatrixScale); }),
+                    new STFReader.TokenProcessor("esd_ortsmatrixrotation", ()=>{ ParseMatrixOverride(STFReader.UNITS.Angle, stf, ref ESD_MatrixRotation); }),
                 });
-                // TODO - some objects have no bounding box - ie JP2BillboardTree1.sd
-                //if (ESD_Bounding_Box == null) throw new STFException(stf, "Missing ESD_Bound_Box statement");
+
+                // Store set of all matrices that got modified
+                foreach (string mat in ESD_MatrixRename.Keys)
+                    ESD_ModifiedMatrices.Add(mat);
+                foreach (string mat in ESD_MatrixTranslation.Keys)
+                    ESD_ModifiedMatrices.Add(mat);
+                foreach (string mat in ESD_MatrixScale.Keys)
+                    ESD_ModifiedMatrices.Add(mat);
+                foreach (string mat in ESD_MatrixRotation.Keys)
+                    ESD_ModifiedMatrices.Add(mat);
             }
             public int ESD_Detail_Level;
             public int ESD_Alternative_Texture;
@@ -96,6 +110,48 @@ public SDShape(STFReader stf)
             public bool ESD_SubObj;
             public string ESD_SoundFileName = "";
             public float ESD_CustomAnimationFPS = 8;
+            // Dictionary of <original texture name, replacement texture name>
+            public Dictionary<string, string> ESD_TextureReplacement = new Dictionary<string, string>();
+            // Dictionary of <original matrix name, replacement matrix name>
+            public Dictionary<string, string> ESD_MatrixRename = new Dictionary<string, string>();
+            // Set of matrix names that are modified in some way or another
+            public HashSet<string> ESD_ModifiedMatrices = new HashSet<string>();
+            // Dictionary of <matrix name, matrix translation x/y/z vector>
+            public Dictionary<string, Vector3> ESD_MatrixTranslation = new Dictionary<string, Vector3>();
+            // Dictionary of <matrix name, matrix scale x/y/z vector>
+            public Dictionary<string, Vector3> ESD_MatrixScale = new Dictionary<string, Vector3>();
+            // Dictionary of <matrix name, matrix rotation x/y/z vector>
+            public Dictionary<string, Vector3> ESD_MatrixRotation = new Dictionary<string, Vector3>();
+
+            // Handle parameters concerning replacement of string values
+            protected void ParseReplacementStrings(STFReader stf, ref Dictionary<string, string> renamePairs)
+            {
+                stf.MustMatch("(");
+                // Allow for multiple pairs of replaced and replacement values
+                while (!stf.EndOfBlock())
+                {
+                    string replaced = stf.ReadString();
+                    string replacement = stf.ReadString();
+                    // Add pair of values so long as we haven't reached the end of block
+                    if (replaced != ")" && replacement != ")")
+                        renamePairs.Add(replaced, replacement);
+                }
+            }
+
+            // Handle matrix adjustment parameters
+            protected void ParseMatrixOverride(STFReader.UNITS units, STFReader stf, ref Dictionary<string, Vector3> matrixParams)
+            {
+                Vector3 data = new Vector3(0);
+
+                stf.MustMatch("(");
+                string matName = stf.ReadString();
+                data.X = stf.ReadFloat(units, 0);
+                data.Y = stf.ReadFloat(units, 0);
+                data.Z = stf.ReadFloat(units, 0);
+                stf.SkipRestOfBlock();
+
+                matrixParams.Add(matName, data);
+            }
         }
 
         public class ESD_Bounding_Box

Source/Orts.Formats.Msts/SoundManagmentFile.cs

@@ -21,6 +21,7 @@
 using System.Diagnostics;
 using System.Linq;
 using System.Text;
+using Microsoft.Xna.Framework;
 using Orts.Parsers.Msts;
 
 namespace Orts.Formats.Msts
@@ -180,6 +181,8 @@ public class SMSStream
         public bool[] Weather;
         public int[] TimeInterval;
         public List<int[]> TimeIntervals;
+        public Vector3 Position; // TODO: Implement user inputs for positional offset and shape hierarchy
+        public int ShapeHierarchy;
 
 
 

Source/Orts.Formats.OR/ContainerFile.cs

@@ -62,7 +62,8 @@ bool TryParse(JsonReader item)
     public class ContainerParameters
     {
         public string Name;
-        public string ShapeFileName;  
+        public string ShapeFileName;
+        public string ShapeDescriptor;
         public string ContainerType;  
         public Vector3 IntrinsicShapeOffset = new Vector3(0f, 1.17f, 0f);
         public float EmptyMassKG = -1;
@@ -78,7 +79,12 @@ bool TryParse(JsonReader item)
             switch (item.Path)
             {
                 case "Name": Name = item.AsString(""); break;
-                case "Shape": ShapeFileName = item.AsString(ShapeFileName); break;
+                case "Shape":
+                    ShapeFileName = item.AsString(ShapeFileName);
+                    if (string.IsNullOrEmpty(ShapeDescriptor))
+                        ShapeDescriptor = ShapeFileName + "d";
+                    break;
+                case "ShapeDescriptor": ShapeDescriptor = item.AsString(ShapeDescriptor); break;
                 case "ContainerType": ContainerType = item.AsString("40ftHC"); break;
                 case "IntrinsicShapeOffset[]": IntrinsicShapeOffset = item.AsVector3(Vector3.Zero); break;
                 case "EmptyMassKG": EmptyMassKG = item.AsFloat(-1); break;

Source/Orts.Simulation/Simulation/Container.cs

@@ -64,6 +64,7 @@ public enum Status
 
         public string Name;
         public string ShapeFileName;
+        public string ShapeDescriptor;
         public string BaseShapeFileFolderSlash;
         public float MassKG = 2000;
         public float EmptyMassKG;
@@ -115,6 +116,7 @@ public virtual void Copy(Container containerCopy)
             Name = containerCopy.Name;
             BaseShapeFileFolderSlash = containerCopy.BaseShapeFileFolderSlash;
             ShapeFileName = containerCopy.ShapeFileName;
+            ShapeDescriptor = containerCopy.ShapeDescriptor;
             IntrinsicShapeOffset = containerCopy.IntrinsicShapeOffset;
             ContainerType = containerCopy.ContainerType;
             ComputeDimensions();
@@ -131,6 +133,7 @@ public Container(BinaryReader inf, FreightAnimationDiscrete freightAnimDiscrete,
             Name = inf.ReadString();
             BaseShapeFileFolderSlash = inf.ReadString();
             ShapeFileName = inf.ReadString();
+            ShapeDescriptor = inf.ReadString();
             LoadFilePath = inf.ReadString();
             IntrinsicShapeOffset.X = inf.ReadSingle();
             IntrinsicShapeOffset.Y = inf.ReadSingle();
@@ -231,6 +234,7 @@ public void Save(BinaryWriter outf, bool fromContainerStation = false)
             outf.Write(Name);
             outf.Write(BaseShapeFileFolderSlash);
             outf.Write(ShapeFileName);
+            outf.Write(ShapeDescriptor);
             outf.Write(LoadFilePath);
             outf.Write(IntrinsicShapeOffset.X);
             outf.Write(IntrinsicShapeOffset.Y);
@@ -253,8 +257,9 @@ public void LoadFromContainerFile(string loadFilePath, string baseFolder)
             var containerFile = new ContainerFile(loadFilePath);
             var containerParameters = containerFile.ContainerParameters;
             Name = containerParameters.Name;
-           
+
             ShapeFileName = @"..\" + containerParameters.ShapeFileName;
+            ShapeDescriptor = @"..\" + containerParameters.ShapeDescriptor;
             var workingString = containerParameters.ShapeFileName.Replace(@"\" , @"/");
             var root  = workingString.Substring(0, workingString.IndexOf(@"/"));
             BaseShapeFileFolderSlash = baseFolder + root + @"\";

Source/Orts.Simulation/Simulation/RollingStocks/Coupling/AnimatedAirHose.cs

@@ -1,4 +1,4 @@
-// COPYRIGHT 2022 by the Open Rails project.
+// COPYRIGHT 2022 by the Open Rails project.
 // 
 // This file is part of Open Rails.
 // 
@@ -32,5 +32,6 @@ public struct AnimatedAirHose
     public struct AnimatedAirHoseState
     {
         public string ShapeFileName;
+        public string ShapeDescriptor;
     }
 }

Source/Orts.Simulation/Simulation/RollingStocks/Coupling/AnimatedCoupler.cs

@@ -1,4 +1,4 @@
-// COPYRIGHT 2022 by the Open Rails project.
+// COPYRIGHT 2022 by the Open Rails project.
 // 
 // This file is part of Open Rails.
 // 
@@ -30,5 +30,6 @@ public struct AnimatedCoupler
     public struct AnimatedCouplerState
     {
         public string ShapeFileName;
+        public string ShapeDescriptor;
     }
 }