SBM MODSCRIPT, PART 18 - USING RANGE FOR EASY LIST INTERACTION

ChaiScript uses the "Range" object for iterating lists, Vectors, and Maps. Mainly, it does it behind the scenes while you use the for ( rec : list ) syntax. However, a Range can be a very useful way to quickly grab the first or last item in a list:

var firstField = range( rec.Fields() ).front();
var lastField = range( rec.Fields() ).back();

The above code creates a Range which wraps the list returned by rec.Fields().  The range can be used for iterating the list if desired, but the helpful hint here is how to access the first or last item in the list. 

 

SBM ModScript - Table of Contents

Tags:
Continue reading
145 Hits
0 Comments

SBM MODSCRIPT, PART 17 - File Fields

File and URL fields have been added to a recent version of SBM. As such, each field can store one or more entries, either files or URLs. The ModScript interface has not yet been updated to make interaction with these fields simple. However, reading from the fields is not all that difficult, as ModScript has access to the TS_FILE_OBJS table, where all the data is stored. Accessing file contents can be more difficult, as the files can either be on the file system or stored as a blob (this is a system configuration option). Also, interacting with files isn't always easy anyways, as files can contain binary data which would make reading the file or changing the file into questionable use cases. However, I was recently asked how a customer could access file data from a file field, where all files are JSON data. This makes sense, we can do something at the script level with JSON data, so I went down the rabbit hole and came up with the following two options. Keep in mind, if you only require the file names and sizes, there is no need to get the file contents. If you do not need the file contents, simply read the rows from TS_FILE_OBJS and decide what to do with the values.

 

Option 1: Direct access from ModScript

ModScript is fairly powerful, we should be able to simply grab the files, consume their contents, and process the JSON at will. For the most part, we can. The big blocker I ran into is that file attachments, when stored on the file system, can be on a network share and require a certain authenticated user to access them. If you are in this configuration, I'd suggest moving on to Option 2, as I did not find any workaround for network user impersonation in ModScript. Also, this doesn't have to be a network folder, it is possible that it is a local folder that requires specific user permissions. In either case, Option 2 is for you. Option 1 works well with files stored as blobs in the database or files stored with no user authentication requirements.

For direct access, ModScript can read the rows from the TS_FILE_OBJS table. It then iterates the entries, and reads the file contents from the related blob or file. It parses the file contents into a JSON object. At that point, it is ready for whatever JSON processing you wish to do. I simply invoked processJSONObj() from part 14 of this series as an example. 

include("processJSONObj");

def AreAttachmentsStoredInDB() {
  global __AreAttachmentsStoredInDB__; // singleton, init once, access only through this function
  if ( __AreAttachmentsStoredInDB__.is_var_undef() ) {
    var setting = Ext.CreateAppRecord( Ext.TableId("TS_SYSTEMSETTINGS") );
    setting.Read("StoreAttachmentsInDatabase");
    __AreAttachmentsStoredInDB__ = 0 != setting.GetFieldValueInt("LONGVALUE");
  }
  return __AreAttachmentsStoredInDB__;
}

def AttachmentsFileSystemLocation() {
  global __AttachmentsFileSystemLocation__; // singleton, init once, access only through this function
  if ( __AttachmentsFileSystemLocation__.is_var_undef() ) {
    var setting = Ext.CreateAppRecord( Ext.TableId("TS_SYSTEMSETTINGS") );
    setting.Read("WorkCenterAttachDir");
    __AttachmentsFileSystemLocation__ = setting.GetFieldValueString("STRINGVALUE");
  }
  return __AttachmentsFileSystemLocation__;
}

class FileFieldEntry {
  var name;
  var filename;
  var fileSystemName;
  var blobID;
  
  def FileFieldEntry() {
    this.name = "";
    this.filename = "";
    this.fileSystemName = "";
    this.blobID = 0;
  }
}

def GetFileFieldEntries( item, fieldName, outVect ) {
  var fileObjs = Ext.CreateAppRecordList( Ext.TableId("TS_FILE_OBJS") );
  var field = item.Fields().FindField( fieldName );
  fileObjs.ReadWithWhere( "TS_RECORDID=? and TS_TABLEID=? and TS_FIELDID=?",
                          [
                            Pair( DBTypeConstants.INTEGER, item.GetId() ),
                            Pair( DBTypeConstants.INTEGER, item.GetRecTableId() ),
                            Pair( DBTypeConstants.INTEGER, field.GetId() )
                          ] );
  for ( fileObj : fileObjs ) {
    var entry = FileFieldEntry();
    entry.name = fileObj.GetFieldValueString("NAME");
    entry.filename = fileObj.GetFieldValueString("FILENAME");
    entry.fileSystemName = fileObj.GetFieldValueString("CONTENTS");
    entry.blobID = fileObj.GetFieldValueInt64("BLOBID");
    outVect.push_back( entry );
  }
}

def GetFileContents( FileFieldEntry entry ) {
  if ( AreAttachmentsStoredInDB() ) {
    var f = TempFile();
    Shell.Db().WriteBlobToFile( entry.blobID, f.GetFileName() );
    return Ext.ReadTextFile( f.GetFileName() );
  }
  else {
    var path = AttachmentsFileSystemLocation();
    path += '\\';
    path += entry.fileSystemName;
    return Ext.ReadTextFile( path );
  }
}

var fileFieldEntries = [];
GetFileFieldEntries( Shell.Item(), "JSON_FILES", fileFieldEntries );
for ( fileFieldEntry : fileFieldEntries ) {
  var fileContents = GetFileContents( fileFieldEntry );
  Ext.WriteStream( fileContents );
  var json = fileContents.from_json();
  processJSONObj( json );
}

 

Option 2: Access file/url field values via REST call to JSONAPI

In many ways, this option is far simpler than Option 1, as you don't have to do it yourself. All we need to do is take advantage of the JSONAPI "GetFileField" function, which will provide the full file contents for our field. The one part that made this hard was that the JSONAPI uses BASE64 encoding on the file contents, and I needed some way to decode the contents as text so that I could work with it. This is why I wrote Part 16, where I provided a way to decode BASE64 values to text. First, you will need to add a RESTDataSource to your process app. Call it "SBM_JSONAPI" and point it to "http://localhost/workcenter/tmtrack.dll?JSONPage&command=jsonapi&JSON_Func=". It may be helpful to create an Endpoint for this, which allows you to customize the URL in AR for various environments. When ModScript invokes this URL, it needs to be able to get in to SBM AE, so the url may need to be https and could need to point to a specific AE runtime server or load balancer. The point here is that ModScript is invoking SBM AE's REST JSONAPI, so the URL has to help ModScript get there. The authentication type "Security Token" will probably work for handling auth, but this may also need to be fiddled with.

Once you have a RESTDataSource that will be used by ModScript, we can proceed:

include("processJSONObj");

def GetFileFieldEntries( item, fieldName ) {
  var REST = Ext.CreateAppRecord( Ext.TableId( "TS_RESTDATASOURCE" ) );
  REST.Read("SBM_JSONAPI");
  var out = "";
  if ( !REST.Get( out, [ 
                         Pair( "JSON_Func", "GetFileField" ),
                         Pair( "JSON_P1", item.GetRecTableId() ),
                         Pair( "JSON_P2", item.GetId() ),
                         Pair( "JSON_P3", fieldName )
                       ] ) ) {
    Ext.WriteStream( Shell.GetLastErrorMessage() );
    ExitScript();
  }
  return out;
}

add_global_const("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/", "CONST_BASE64TABLE");
def Base64DecodeAsText( string input ) { // assumes output is valid text (not binary)
  var sOut = "";
   
  var buf = [uint8_t(),uint8_t(), uint8_t(), uint8_t()];
  var encoded = int( input.size() );
  var count = 3 * ( encoded / 4 );
  var i = 0;
  var j = 0;
  
  while ( sOut.size() < count ) {
    // Get the next group of four characters
    //    'xx==' decodes to  8 bits
    //    'xxx=' decodes to 16 bits
    //    'xxxx' decodes to 24 bits
    for_each( buf, fun( entry ){ entry = 0; } ); // zero out buffer
    var stop = min( encoded - i + 1, 4 );
    for ( j = 0; j < stop; ++j ) {
      if ( input[i] == '=' ) {
        // '=' indicates less than 24 bits
        buf[j] = 0;
        --j;
        break;
      }

      // find the index_of inside CONST_BASE64TABLE for our value
      buf[j] = fun( s, c ) {
        for ( var i = 0; i < s.size(); ++i ) {
          if ( s[i] == c ) {
            return i;
          }
        }
        return string_npos;
      }( CONST_BASE64TABLE, input[i] );
      ++i;
    }
	
    // Assign value to output buffer
    sOut += char(buf[0] << 2 | buf[1] >> 4);
    if ( sOut.size() == count || j == 1 ) {
      break;
    }
    
    sOut += char(buf[1] << 4 | buf[2] >> 2);
    if ( sOut.size() == count || j == 2 ) {
      break;
    }
	
    sOut += char(buf[2] << 6 | buf[3]);
  }
  
  return sOut;
}

var fileFieldValue = GetFileFieldEntries( Shell.Item(), "JSON_FILES" );
var fileFieldValueJSON = fileFieldValue.from_json()["fieldFileObj"]["fileObjList"];
for ( fileFieldEntry : fileFieldValueJSON ) {
  var fileContents = Base64DecodeAsText( fileFieldEntry["contentsBase64"]["data"] );
  var json = fileContents.from_json();
  processJSONObj( json );
}

 

SBM ModScript - Table of Contents

Tags:
Continue reading
149 Hits
0 Comments

SBM MODSCRIPT, PART 16 - BASE64DECODE

I recently discovered that I would need a way to do base64 decoding for a ModScript I was writing. This can be tricky, as the output could be a binary value with embedded zeros. You could certainly do this with the output as a Vector with each entry a uint8_t (unsigned byte). However, in my use case, I knew that the data was text and could be represented as a string. As such, I wrote the following:

add_global_const("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/", "CONST_BASE64TABLE");
def Base64DecodeAsText( string input ) { // assumes output is valid text (not binary)   
  var sOut = "";
   
  var buf = [uint8_t(),uint8_t(), uint8_t(), uint8_t()];
  var encoded = int( input.size() );
  var count = 3 * ( encoded / 4 );
  var i = 0;
  var j = 0;
  
  while ( sOut.size() < count ) {
    // Get the next group of four characters
    //    'xx==' decodes to  8 bits
    //    'xxx=' decodes to 16 bits
    //    'xxxx' decodes to 24 bits
    for_each( buf, fun( entry ){ entry = 0; } ); // zero out buffer
    var stop = min( encoded - i + 1, 4 );
    for ( j = 0; j < stop; ++j ) {
      if ( input[i] == '=' ) {
        // '=' indicates less than 24 bits
        buf[j] = 0;
        --j;
        break;
      }

      // find the index_of inside CONST_BASE64TABLE for our value
      buf[j] = fun( s, c ) {
        for ( var i = 0; i < s.size(); ++i ) {
          if ( s[i] == c ) {
	        return i;
	      }
        }
        return string_npos;
      }( CONST_BASE64TABLE, input[i] );
      ++i;
    }
	
    // Assign value to output buffer
    sOut += char(buf[0] << 2 | buf[1] >> 4);
    if ( sOut.size() == count || j == 1 ) {
      break;
    }
    
    sOut += char(buf[1] << 4 | buf[2] >> 2);
    if ( sOut.size() == count || j == 2 ) {
      break;
    }
	
    sOut += char(buf[2] << 6 | buf[3]);
  }
  
  return sOut;
}

The function above iterates the input string contents and uses base64 to create an decoded output string. Notice that the "buf" variable is a Vector of 4 unsigned, 8 bit integers. As we are going to use bit shifting in order to decode the data, it is important to use unsigned byte data to ensure the expected bit-shift result. We find the index of the character in CONST_BASE64TABLE to find the data-representation we are looking for, then use bit shifting to convert the buf value to text. The result is the original text after processing the base64 algorithm. A possible use case for this might be in decoding HTTP headers from a REST call. 

SBM ModScript - Table of Contents

Tags:
Continue reading
115 Hits
0 Comments

How to view a ModScript log file without using remote desktop connection

ModScript, at least since version 11.3, added a new "Log" object with methods to allow sending text to a log file in the SBM "Application Engine\Log" directory.  To view these messages you need to start a "Remote Desktop Connection", aka "Remote Console" or RDP to the AE server and view the log file in the "Application Engine\log" directory.  Previously, this kind of logging was done using the "Ext.LogInfoMsg" call which sent the message to the AE server's Windows Application Event log.  To view these log messages, you also had to "remote" into the AE server, then use the Windows Event Viewer to view the Application Event log.  That level of access can make some clients or server administrators nervous.  In some conditions, the developer may not have any access to SBM servers.

Fortunately, there's a way to create a log that doesn't require any remote access to the AE server, outside of IIS.

Some basic "one time" setup is necessary to allow this capability. Have someone with "administrator" privileges on the SBM AE server do the following steps:

  1. "remote" into the SBM AE server
  2. "CD" to IIS's "inetpub\wwwroot" directory.
  3. Create a subdirectory under "inetpub\wwwroot" for the ModScript log files.  In this example the name of the subdir is "AE-Logs".
  4. Use the Microsoft Sys Internals "junction" tool "junction" to create a file junction from "inetpub\wwwroot\AE-Logs" that is targeted at a subdirectory (which the "junction" command will create) under the App Engine's "Log" directory.  In this example the name of the subdir under "Application Engine\Log" is called "inetpub_AE-logs".

junction  "C:\inetpub\wwwroot\AE-logs"   "F:\Program Files\Serena\SBM\Application Engine\Log\inetpub_AE-logs"

Now your ModScript can use the following calls to create a Log file in that new subdir.  Make sure that the name of the log file you create has a file extension that IIS will handle, like ".txt" or ".htm".  On my system, IIS will not handle a file with the ".log" extension.  I can probably change that by making changes to the IIS settings.

var my_IIS_log Log() ;
my_IIS_log.Open("inetpub_AE-logs/ModScript_log.txt");
my_IIS_log.SetWantTimeStamp(true);
my_IIS_log.Message( LogLevelConstants.AVERAGE, "0 param call to my_IIS_log.Message" );
my_IIS_log.Message( LogLevelConstants.AVERAGE, "3 param call to my_IIS_log.Message : Reporting Level={0} : File={1} : IsOpen={2}" , my_IIS_log.GetReportingLevel() , my_IIS_log.GetFileName() , my_IIS_log.IsOpen() );

 

View the log in a browser:

https://-MY-AE-SERVER-/AE-logs/ModScript_log.txt

Continue reading
417 Hits

SBM ModScript, Part 12 - Class Inheritance

After looking into ChaiScript classes, I discovered that ChaiScript does not currently support class inheritance. This is common for scripting languages. However, as ChaiScript is such a powerful language, I was wondering how hard it would be to implement it. I came up with the following two options.

Option 1

Option 1 was updated on June 11, 2018 to introduce the "this.This" object, which allows for methods in the base class to invoke overrides on the child class.

def _getBaseVarName( obj ) {
  return "_${obj.get_type_name()}_base";
}

def _getBase( obj ) {
  return eval("obj.${_getBaseVarName(obj)}");
}

def _setThis( obj, ancestor ) {
  ancestor.This := obj;
  if ( eval( "!ancestor.${_getBaseVarName(ancestor)}.is_var_undef()" ) ) {
    _setThis( obj, ancestor._getBase() );
  }
}

def _addMethod( obj, funcname, containedFunc ) {
  var pts = containedFunc.get_param_types();
  
  // since we checked arity above, we know there will be at least 2 entries, so this is safe
  if ( pts[1].is_type_undef() || pts[1].name() == "Dynamic_Object" ) { // if parameter is for a Dynamic_Object class, this is true
    // ensure there are no guards on the parameters (pts[0] is return, pts[1] is "this"
    if ( pts.size() > 2 ) {
      for( var ii = 2; ii < pts.size(); ++ii ) {
        if ( pts[ii].name() != "Object" ) {
          return false;
        }
      }
    }
    
    var paramStr = ""
    for ( var ii = 1; ii < containedFunc.get_arity(); ++ii ) {
      if ( ii != 1 ) {
        paramStr += ',';
      }
      paramStr += "v${ii}";
    }
    eval( "def ${obj.get_type_name()}::${funcname}(${paramStr}){ this._getBase().${funcname}(${paramStr}); }" );
    return true;
  }
  return false;
}

def _verifyAndAddMethod( obj, base, funcname, func ) {
  var sParamVals = "";
  for ( var x = 1; x < func.get_arity(); ++x ) {
    sParamVals += ",0";
  }
  
  if ( eval("call_exists( func, base ${sParamVals} );") && // method exists on base object
       !eval("call_exists( func, obj ${sParamVals} );") && // method does not exist on this object (override)
       !func.has_guard() &&                                // method may not have guards
       !func.get_contained_functions()[0].has_guard() &&   // method may not have guards
       funcname != "This" &&                               // ignore "This" attribute
       !_addMethod( obj, funcname, func ) ) {
    for ( containedFunc : func.get_contained_functions() ){
      if ( !containedFunc.get_contained_functions().empty() && _addMethod( obj, funcname, containedFunc ) ) {
        break;
      }
    }
  }
}

def _setup( obj ) {
  _setThis( obj, obj );
}

def _inherit( obj, base ) {
  obj._getBase() = base;
  _setup( obj );
  
  global _inheritance;
  if ( _inheritance.is_var_undef() ) {
    _inheritance = Map();
  }
  
  // only set up this class inheritance once
  if ( _inheritance.count( obj.get_type_name() ) == 0 ) {
    _inheritance[obj.get_type_name()] = base.get_type_name();
    
    // must invoke get_functions every time, cannot filter and save list, as class definitions might occur after we 
    // hit this for the first time
    for ( func : get_functions() ) {
      if ( func.second.get_arity() == -1 ) {
        for ( inner : func.second.get_contained_functions() ) {
          if ( inner.get_arity() > 0 && !inner.get_contained_functions().empty() ) {
            _verifyAndAddMethod( obj, base, func.first, inner );
          }
        }
      }
      else if ( func.second.get_arity() > 0 && !func.second.get_contained_functions().empty() ) {
        _verifyAndAddMethod( obj, base, func.first, func.second );
      }
    }
  }
}
Usage:
class Base {
  attr id;
  def Base() { _setup( this ); }
  def Base( ID ) { _setup( this ); this.id = ID; }
  def go( x ){ print("go(x) | dynamicType: ${this.This.get_type_name()} | x: ${x} | id: ${this.This.id}"); }
  def go2( x ){ print( "go2(x) | hardcodedType: Base | x: ${x} | id: ${this.This.id}" ); }
  def go3( x, y ){ print( "go3(x,y) | hardcodedType: Base | x: ${x} | x: ${y} | id: ${this.This.id}" ); }
};

class Child {
  def Child() { 
    _inherit( this, Base() );
  }
  
  def Child( ID ) {
    _inherit( this, Base( ID ) );
  }
  
  // override that (optionally) invokes base
  def go2( x ){ print( "Child: ${x}: ${this.This.id}" ); this._getBase().go2( x ); }
};

class GrandChild {
  def GrandChild() { 
    _inherit( this, Child() );
  }
  
  def GrandChild( ID ) {
    _inherit( this, Child( ID ) );
  }
  
  // override that (optionally) invokes _base
  def go3( x, y ){ print( "go3(x,y) | dynamicType: ${this.get_type_name()} | x: ${x} | x: ${y}" );  this._getBase().go3( x,y ); }
};
What Does It Do?
  • In any given base class (class which does not inherit from another class, but will be inherited from), invoke "_setup(this);".
    • This will set up a "this.This" attribute on the object.
  • In the constructors of child classes, invoke "_inherit( this, )".
    • The class of the base object is used to determine the parent class of this object. It is important to use the same base class for every constructor for the child class. 
    • You can invoke any of the parent's constructors, allowing you to initialize the parent object as desired.
  • Behind the scenes of the _inherit() function
    • It will look at all the attributes and methods of the parent class and, if it doesn't find an override in the child class, it will define a method in the child class that invokes the method from the parent class. As such, any given child class object will have access to all the attributes and methods of the parent object.
    • The base object passed to _inherit is stored as a member on the child object. It is accessible via the "this._getBase()" method. This allows you to invoke the parent class method directly if desired.
    • Each object of type GrandChild will look like this (simplified):
      • GrandChild class
        • this = theGrandChildObj
        • this.This = theGrandChildObj
        • this._getBase() = theChildObj (Child class)
          • this = theChildObj
          • this.This = theGrandChildObj
          • this._getBase() = theBaseObj (Base class)
            • this = theBaseObj
            • this.This = theGrandChildObj
            • this._getBase() = undefinedObj
    • As you can see, in the GrandChild constructor, we create an object of type Child (which gets stored in the "this._getBase()" location). The Child constructor creates an object of type Base which gets stored in the Child object's "this._getBase()" location. Therefore, in any given method, you can call the parent object's version of the method by invoking this._getBase().method().
    • Each object will point to itself via the "this" attribute. Be careful when you use "this", as it is safer to use "this.This" to ensure polymorphic overrides are invoked.
    • Each object will point to the actual object via the "this.This" attribute. As such, any given method can polymorphically invoke the methods of the child object.
      • Do NOT use "this.This" when invoking "this._getBase()".
  • Watch out for guards on methods or parameters. It is just too complex to support class method overrides with guards, so they are not supported in this implementation. Also, each class is still separate as far as the ChaiScript class engine is concerned, so a function like "foo( Base bar )" will not recognize an object of type "Child". As long as you leave the guards out, all objects should act polymorphically correct.
Notes
  • Make sure to invoke _inherit() in every constructor on the child class, and pass in an object from the parent class.
  • Make sure to invoke _setup() in every class intended for usage as a base class. This will set up the "this.This" pointer, which allows the base class methods to invoke the overridden methods in the child class.
  • Attributes and Methods added to the parent class after a child class object has been created will never be visible to objects of the child class.
  • No classes in the class hierarchy may have methods with guards on the parameters or on the functions.
  • Objects of the child class will not convert to the base class, do not try to pass an object of type Child to a function like foo( Base bar ).
  • When invoking a method from the base class, code inside the base class method will treat the "this" object as if it is of the base class. Calls in the base method to other methods that have overrides in the child will not invoke the child override if you use the "this" object. To get the child methods to invoke from base methods, use the "this.This" object.
  • Classes built in to ModScript cannot be inherited from.

 

Option 2

There is another option, which has different benefits and drawbacks. The biggest drawback is that you cannot use the ChaiScript "class" syntax.

global _classInheritence_ = ["VirtualClass" : ""];
global _vtable_ = Map();

// All "classes" will be instances of VirtualClass, but with a different
// value for _classname_. It knows how to look up the stuff for the
// fake "class".
class VirtualClass{
  attr _classname_;
  def VirtualClass( sClass ) { this._classname_ = sClass; }
  
  // a method for invoking the function in the parent class
  def _callBase( classname, funcName, params ) {
    // verify "classname" is base of "this"
    var classnameSearch = this._classname_;
    while ( !classnameSearch.empty() ) {
      if ( classnameSearch == classname ) {
        break;
      }
      classnameSearch = _classInheritence_[classnameSearch];
    }
    if ( classnameSearch.empty() ) {
      Ext.LogErrorMsg( __FILE__ + ":" + __LINE__ + ": could not find base class ${classname} for ${this._classname_}");
      throw(0);
    }
    var f = _findVMethod( "${funcName}:${params.size()+1}", classname );
    var p2 = [this];
    params.for_each( back_inserter( p2 ) );
    return call( f, p2 );
  }
}

def _findVMethod( sFuncWithArrity, classname ) {
  var virtFuncArrity = _vtable_.at( sFuncWithArrity );
  var classnameSearch = classname;
  while ( !classnameSearch.empty() ) {
    if ( virtFuncArrity.count( classnameSearch ) == 1 ) {
      return virtFuncArrity[ classnameSearch ];
    }
    classnameSearch = _classInheritence_[classnameSearch];
  }
  Ext.LogErrorMsg( __FILE__ + ":" + __LINE__ + ": could not find function ${sFuncWithArrity} for class ${classname}" );
  throw(0);
}

def VClass( sClass, sParentClass ) {
  if ( _classInheritence_.count(sClass) == 0 ) {
    //eval( "global ${sClass} = fun(){ return VirtualClass( \"${sClass}\" ); };" );
    eval( "def ${sClass}(){ return VirtualClass( \"${sClass}\" ); }" );
    _classInheritence_[sClass] = sParentClass;
  }
}

def VClass( sClass ) {
  VClass( sClass, "VirtualClass" );
}

def VMethod( sClass, sFunc, f ) {
  if ( f.get_arity() < 1 ) {
    return;
  }
  else if ( f.has_guard() ) {
    Ext.LogErrorMsg( __FILE__ + ":" + __LINE__ + ": registerVirtual failed for function ${sFunc}, virtual function cannot have guard" );
    throw(0);
  }
  else {
    // check the params of f, reject if any params have a guard
    for ( p : f.get_param_types() ){
      if ( p.name() != "Object" ) {
        Ext.LogErrorMsg( __FILE__ + ":" + __LINE__ + ": registerVirtual failed for function ${sFunc}, virtual function cannot have guard on parameter" );
        throw(0);
      }
    }
  }
  
  var sFuncWithArrity = "${sFunc}:${f.get_arity()}";
  
  if ( _vtable_.count( sFuncWithArrity ) == 0 ) {
    _vtable_[sFuncWithArrity] = Map();
    var sParams = "";
    for ( var v = 1; v < f.get_arity(); ++v ) {
      if ( v != 1 ) {
        sParams += ',';
      }
      sParams += "v${v}";
    }
    
    // add a method to the VirtualClass that will look up the method for this "class"
    eval( 
"def VirtualClass::${sFunc}(${sParams}){ 
  /* _findVMethod throws if not found */
  var f = _findVMethod( \"${sFuncWithArrity}\", this._classname_ );
  return f( this ${sParams.empty() ? "" : ","} ${sParams} );
}" );
  }
  
  _vtable_[sFuncWithArrity][sClass] = f;
}

def VClassAttr( sClass, sMember ) {
  VMethod( sClass, sMember, eval( "fun(this){ return this._${sClass}_${sMember}; }" ) );
}
Usage:
VClass( "Base" );
VMethod( "Base", "go", fun(this, x){ print( "${this._classname_}: ${x}" ); } );
VMethod( "Base", "go2", fun(this, x){ print( "Base: ${x}" ); } );
VClassAttr( "Base", "id" );

VClass( "Child", "Base" );
VMethod( "Child", "go2", fun(this, x){ 

// optional call to base
this._callBase( "Base", "go2", [x] );
} ); VClass( "GrandChild", "Child" ); VMethod( "GrandChild", "go2", fun(this, x){ this._callBase( "Base", "go2", [x] ); } );

 

Notes
  • All objects are instances of the VirtualBase class
    • All methods and attributes for each class are added to the _vtable_ Map.
    • Each object has an attribute that tells it which class it is.
    • Function calls will look up the method for the current class, and if no override is found, will search up the class tree to find the method or attribute.
    • As opposed to Option 1, when invoking a method from the base class, code inside the base class method will treat the object as if it is of the child class. Calls in the base method to other methods that have overrides in the child will invoke the child override. As such, there is no need for a "this.This" attribute.
  • No virtual methods may have methods with guards on the parameters or on the functions.
  • All objects are of type VirtualBase, do not try to pass an object of type Child to a function like foo( Base bar ), but you can pass any objects from this hierarchy to a function like  foo( VirtualBase bar ).
  • Classes built in to ModScript cannot be inherited from.
  • Option 2 is faster than Option 1.
  • this._callBase() requires that you tell it the name of the base class, and it checks to ensure that the base class you select is an ancestor of the current class.

 

SBM ModScript Blog Series

Continue reading
1760 Hits
0 Comments

SBM ModScript Blog Series

SBM ModScript was introduced in SBM 11.3. This blog series is intended to supplement the SBM ModScript Guide with background information and detailed use cases.  You can find the latest version of the documentation here

  1. An Introduction to ModScript
    • Background on SBM ModScript
  2. Transition Related Items
    • An example script containing several key features of SBM ModScript, including:
      • Function definitions
      • Regular expressions
      • List iteration
      • Defining constants
      • Transitioning SBM items
      • Reading lists of SBM items
      • Using the from_json() function
      • Multi-line comments
  3. Adding Methods to a Class
    • How to use the script engine's ability to extend classes to add new methods.
  4. JSON
    • How to use the script engine's to_json() and from_json() utility functions.
  5. Algorithms and Lambdas
    • How to use the script engine's built-in algorithms, along with the optional use of lambdas (anonymous functions).
  6. Invoking DLLs
    • How to interact with custom DLLs.
  7. REST Call Into ModScript
    • How to invoke a ModScript via a URL.
  8. REST Callouts
    • How to invoke a REST endpoint from ModScript.
  9. SQL Queries
    • An introduction to the SQL interaction available in ModScript.
  10. Regular Expressions
    • How to use regular expressions in ModScript.
  11. Transitions
    • A review of how to invoke transitions on Primary/Auxiliary items in ModScript
  12. Class Inheritence
    • An experimental approach to class inheritance.
  13. Base64Encode
    • A utility function for encoding a string using base64 
  14. Checking the Type of a Variable
    • Utility functions available for interrogating the type of a var.
  15. Singletons
    • Using the singleton design pattern.
  16. Base64Decode
    • A utility function for decoding a base64 value to text (not binary)
  17. File Fields
    • Getting an item's file field contents.
  18. Using Ranges for Easy List Interaction
    • Quick access to the first or last element in a list.
Recent Comments
Daniel Nolan
x
Friday, 22 June 2018 10:10 AM
Paul Thompson
Yea! I can't explain how frustrated I've been trying to learn ModScript. I find the syntax to be obscure and counter-intuitive, ... Read More
Wednesday, 23 May 2018 4:04 AM
Continue reading
2394 Hits
2 Comments

SBM ModScript, Part 10 - Regular Expressions

ModScript has the ability to execute regular expressions on strings. The interface for this is the Regex class. In the following example, we create a regular expression that will match any string that starts with "t" (the default options make this case-insensitive). We will then read a full list of users and fill a Vector with users whose loginid starts with "t". Finally, we iterate our Vector and write the users we found to the output stream.

def AppRecord::GetLogin() {
	return this.GetFieldValue("LOGINID").to_string();
}

var users = Ext.CreateAppRecordList( Ext.TableId("TS_USERS") );
users.Read();

var regex = Regex();
regex.Compile( "^t" );

var out = [];
filter( users, bind( fun( iuser, innerRegex ){ return innerRegex.Matches( user.GetLogin() ); }, _, regex ), back_inserter( out ) );

for_each( out, fun( user ){ Ext.WriteStream( user.GetLogin() ); } );

Step by step:

  • The first thing should look familiar, we talked about adding methods to an existing class in Part 3. In this case, we are making a function that makes it easy to pull the LOGINID out of the User, getting it as a string. 
  • The call to Ext.CreateAppRecordList(), passing in the return value of a call to Ext.TableId(), should be pretty familiar from previous examples. In this case, we are building a list which will let us read rows from the Users table. Then, we call the AppRecordList.Read() method to read the entire Users table (this might not be a great idea on systems with lots of users, but it works well in my simple example). 
  • We create a Regex() and compile it with a simple "starts with t" rule. Since we pass no options into Regex.Compile(), we get the default, which is case-insensitive.
  • We create an output Vector to hold the records that match our regular expression.
  • We invoke the "filter" algorithm. In Part 5 we talked about algorithms, including filter, bind, and back_inserter.
    • filter() - Loops through the range for container "users", invokes the function passed in, for each object where the function returns true, it invokes the second function.
    • users - this is the container to iterate.
    • bind( fun( user, innerRegex ){ return innerRegex.Matches( user.GetLogin() ); }, _, regex ) - bind returns a function for filter to invoke when iterating the users container
      • When the returned function is invoked by the "filter" algorithm, passing in a user from the users container, that value will be passed to the function as the first parameter, which is indicated by the underscore in the call to bind()
      • Also, bind will pass our regex object to the inner function as the second parameter, indicated by the "regex" after the "_" passed to bind.
      • Finally, the inner function will use the regex to indicate to the "filter" call whether this user matches our regular expression.
    • back_inserter( out ) - Adds the matched values to the "out" Vector
      • When the filter function finds a match, it invokes this function, which will append the user object onto the "out" Vector.
  • We assume you want to do something with the filtered list of users. In this case, I invoke the "for_each" algorithm, which will invoke my lambda function on each item in the Vector. In this case, it will print out the matching users' LOGINIDs.

 

Regular Expressions With Groups

Above, we saw a simple regular expression and a simple call to Regex.Matches(). However, we can also use more complex regular expressions, including group capture.

var regex = Regex();
regex.Compile( "(\\d+)(\\w+)" );

regex.Matches( "123abc" );
for ( var i = 0; i < regex.GroupCount(); ++i ) {
	Ext.WriteStream( regex.GroupVal(i) );
}

In this example, we have a regular expression with two groups. First, we expect 1 or more digits, followed by 1 or more word-characters. When we invoke Regex.Match() on a string, the Regex object will remember the groups that it matched, and you can access them via Regex.GroupCount() and Regex.GroupVal(). Regex.GroupVal( 0 ) will always be the text matched by the entire expression. After that, the rest of the groups will be indexed in the order they were captured in the string. The above example gives the output:

123abc
123
abc

 

Regular Expressions: MatchesAgain

After calling Regex.Matches(), you can continue finding matches by invoking Regex.MatchesAgain(). Below, we'll print out each letter in the string, one by one. The regular expression will match any non-digit (\d), non-non-word (\W) character. We do not need grouping parens because the Regex.GroupVal( 0 ) call will always give us the full string that was matched by the regular expression. One thing to note, ChaiScript does not give us a do-while loop, so instead you'll see a while(true)-if-break loop, which is exactly the same paradigm.

var regex = Regex();
regex.Compile( "[^\\d\\W]" );

if ( regex.Matches( "123abc" ) ) {
  while (true) {
	Ext.WriteStream( regex.GroupVal( 0 ) );
	if ( !regex.MatchesAgain() ) { break; }
  } 
}

 

Regular Expressions: ReplaceAll

Finally, the ModScript Regex class has the ability to use regular expression matching to replace values in strings, returning a modified string with all matches replaced. Also, you can use $ notation to use the matched value, or a matching group number, in the replacement: $0 is the entire matched value, $1 would be the first captured group in the match, etc). 

var regex = Regex();
regex.Compile( "[^\\d\\W]" );

Ext.WriteStream( regex.ReplaceAll( "123abc", "(\$0)" ) );

 

Output:

123(a)(b)(c)

What happened? I replaced each matching value, in this case the a, b, and c, with a value of the matched text wrapped with parenthesis. As such, "a" became "(a)", etc. You do not need to use groups in the replacement value, you can replace each letter with "D" if you wish. Keep in mind that if you are not trying to use a dollar group-identifier in the replacement string, you will want to escape the dollar symbol with a double backslash \\.

 

Notes:
  • It is important to remember that most regular expressions have backslashes in them. ChaiScript uses backslash in string literals to identify special characters like newline: \n and tab: \t. As such, all backslashes that are intended for the regular expression need to be double-backslash \\.
  • As the dollar $ symbol is important in regular expressions, it is important to remember that it also means something in ChaiScript. If ChaiScript finds a ${...} in the string, it will try to invoke the value inside the curly braces as if it were string-injected-script. This could be pretty messy if you accidentally mixed it with a regular expression. It is wise to ChaiScript-escape the $ in the string with a single backslash. If you also are trying to regular-expression-escape the dollar, you may need \\\$.
  • The Regex.Compile() function takes an optional second parameter, which is used to indicate options. The default is a case-insensitive, single-line regular expression. To shut off case-insensitive but keep the other options, pass 0. To set the options you like, pass the options to the second parameter, connected with the ChaiScript bitwise-or operator | . 
    • RegexOptionBitsConstants.IGNORECASE
    • RegexOptionBitsConstants.MULTILINE
    • RegexOptionBitsConstants.DOT_MATCHES_ALL

 

SBM ModScript - Table of Contents

Recent Comments
Paul Thompson
Don: What RE engine does modscript use (i.e. PCRE, Oniguruma, Boost etc). Basically I would like to know the capabilities (non-gr... Read More
Monday, 17 June 2019 1:01 AM
Paul Thompson
Don: Can you confirm that POSIX character classes are supported? I'm having trouble with:var regex = Regex() ; var str_Test = st... Read More
Saturday, 21 September 2019 5:05 AM
Don Inghram
We use PCRE2.
Monday, 17 June 2019 4:04 PM
Continue reading
1550 Hits
4 Comments

SBM ModScript, Part 9 - SQL Queries

In ModScript, we can read queries from the SBM database. One way to do this is to read items from a table, where each item is based on that specific table's schema. This is what the AppRecord, VarRecord, and ProjectBasedRecord classes (and child classes) are for. When you create one with Ext.CreateAppRecord(), Ext.CreateVarRecord(), or Ext.CreateProjectBasedRecord(), you always pass in the table ID that the record will be based on, so for the life of that object, it will be associated with that table and its schema. Also, we have the AppRecordList (and child classes), which is created with Ext.CreateAppRecordList(), and which also will be bound to a specific table ID on creation. Thus, all items in the AppRecordList will be objects of that tables' type.

All these classes can be used to read records from the SBM table for which the class is bound, using the following functions:

  • AppRecord.Read()
    • Read an item by TS_ID or name.
  • AppRecord.ReadByColumn()
    • Read an item by a column from that table's schema
  • AppRecord.ReadByColumnAndColumn()
    • Use two column values to identify the item to read.
  • AppRecord.ReadWithWhere()
    • Read the item with any SQL where clause. This function can use SQL bind parameters.
  • AppRecord.ReadByUUID()
    • Read an item by TS_UUID
  • AppRecordList.Read()
    • Read an entire table, this could potentially use a lot of resources.
  • AppRecordList.ReadByColumn()
    • Read a list of items by a column from that table's schema
  • AppRecordList.ReadByColumnAndColumn()
    • Use two column values to identify the items to read.
  • AppRecordList.ReadWithWhere()
    • Read the list of items with any SQL where clause. This function can use SQL bind parameters.

When using the ReadWithWhere() functions, I'd encourage the usage of SQL bind parameters. When trying to build SQL using dynamic values, you need to worry that the values you are pulling from fields or other sources could have embedded single-quotes ( ticks: ' ), which can break the SQL. These values would need to be encoded if added directly to the SQL query. However, if instead you put a question mark in the SQL, you can bind the value to it using a Vector of bind values. These bind values do not need to have the ticks encoded, which simplifies your script. In Part 2 of this series, you saw me use SQL bind parameters:

containerList.ReadWithWhere(
    "TS_ID in (select TS_SOURCERECORDID from TS_USAGES where TS_FIELDID=? and TS_RELATEDRECORDID=?)",
    [ Pair(DBTypeConstants.INTEGER, relational.GetId()),
      Pair(DBTypeConstants.INTEGER, Shell.Item().GetId()) ] );

Each entry in the Vector is a Pair, with the first value being the data type for the value, and the second value being the value I am binding to my SQL. 

 

AppDb Queries

ModScript also allows for SQL queries that are not tied directly to tables. The Shell.Db() method returns an AppDb object which points to the current SBM AE schema. It has the following functions which allow for free SQL to be executed, (each take an optional Vector of SQL bind parameters):

  • AppDb.ReadIntWithSQL()
    • Returns a single integer read from the database.
  • AppDb.ReadTextWithSQL()
    • Returns a single string read from the database.
  • AppDb.ReadIntegersWithSQL()
    • Fills a Vector with integers read from the database.
  • AppDb.ReadIntegerPairsWithSQL()
    • Fills a Vector with Pairs of integers read from the database.
  • AppDb.ReadTextValsWithSQL()
    • Fills a Vector with strings read from the database.

With these, you can execute any SQL you want as long as the output of the SQL can be bound to as a single int, varchar, a list of ints, list of varchars, or a list of pairs of ints. In 11.4, we added AppDb.ReadDynaSQL(), which allows you to specify any number of columns, with their column types, which makes you truly free to execute any SQL you want.

 

SQL Performance

Keep in mind that queries can be slow. In SBM, we try to identify any slow queries built into our product, and we add database indexes to mitigate the performance slowdown. However, we can't do this for queries that you add, so you will probably want to watch for places where you should add your own indexes to the database to help speed up your queries.

 

SBM ModScript - Table of Contents

Continue reading
1611 Hits
0 Comments

SBM ModScript, Part 8 - REST Callouts

In ModScript, we can make calls to external REST APIs. Being able to pull in data or send data to a REST API really grows the ability to build integrations with ModScript. In my example, I use the experimental SBM feature Data Service. Data Service allows us to create a connection to a database, which can be the current SBM database or any other database via an ODBC DSN, and pre-configure an SQL query that can be requested via REST. The SQL query can have runtime parameters bound to them from URL parameters.

 

SBM Data Service

The SBM Data Service is an experimental feature. As such, it must be enabled in the TS_SYSTEMSETTINGSNAMESPACED table:

update TS_SYSTEMSETTINGSNAMESPACED set TS_LONGVALUE=1 where TS_NAME='EnableDataServices'

Next, edit DataServiceConf.xml in SBM\Application Engine\bin. Read the big comment in the xml file to get more information about the different types of connections you can use in Data Service.  In my example, I create a DSN-based Connection, even though I could just use a local-SBM connection. The reason I do this is because I think it is likely that my reader will really want to pull data from a different database rather than pull data from the SBM database. ModScript has powerful SQL querying features for querying the SBM database, and probably will not need to invoke a REST service to do it. However, ModScript cannot peek into a non-SBM database using its SQL query features; as such, it is a more likely use case to invoke Data Services via REST to query a separate database. However, to keep the example simple, I use the DSN-based Connection to peek back into the SBM database, as I know it is a database that my readers have on-site.

<DataServices>
  <Connection name="ModScriptService" >
    <DSN_String>DSN=SBM_AE;UID=sa;PWD=serena123!</DSN_String>
  </Connection>
  <DataService name="ModScriptQuery1" connection="ModScriptService">
    <columns>
      <column display="ID" column="TS_ID" type="number-integer" />
      <column display="Name" column="TS_TITLE" type="text" len="512" />
      <column display="Submitter" column="TS_SUBMITTER" type="number-integer" />
      <column display="SubmitDate" column="TS_SUBMITDATE" type="datetime" />
    </columns>
    <query>
      <queryText>select TS_ID, TS_TITLE, TS_SUBMITTER, TS_SUBMITDATE from {namespacePrefix}USR_MODSCRIPTRESTCALL {nolockhint} where TS_ACTIVEINACTIVE={0} and TS_SUBMITTER in ({1})</queryText>
      <params>
        <param index="0" type="int" name="active"/>
        <param index="1" type="intArray" name="submitter"/>
      </params>
    </query>
  </DataService>
</DataServices>

In this example, we search the primary table USR_MODSCRIPTRESTCALL for items where TS_ACTIVEINACTIVE is equal to the value passed on the URL, and TS_SUBMITTER is equal to the value passed on the URL. We return the TS_ID, TS_TITLE, TS_SUBMITTER, and TS_SUBMITDATE. This query could be much more complex. It could create a temp table, insert data into it, then join to that data to collate it all into the desired output. However, this example is not about fancy SQL queries, so we keep it simple.

The DataServiceConf.xml file is only processed on AE startup to keep the service fast. Unfortunately, this means IIS has to be reset any time you change the file. So, after an IIS reset, a call to the DataService URL will set up the query, bind the URL parameters to the query, and execute it.

http:///workcenter/tmtrack.dll?jsonpage&command=dataservice&service=ModScriptQuery1&active=0&submitter=8

The example output:

{
	"data": [{
		"ID": 1,
		"Name": "abc",
		"Submitter": 8,
		"SubmitDate": "\/DATE(1525900070000)\/"
	}, {
		"ID": 14,
		"Name": "def",
		"Submitter": 8,
		"SubmitDate": "\/DATE(1525914739000)\/"
	}],
	"result": {
		"type": "OK",
		"msg": ""
	}
}

 

ModScript

In this example, the ModScript will be fairly simple. It requests to data from Data Service and injects it into the HTML form as a JavaScript variable. I added this script as a pre-transition action on the submit transition of my workflow. What this will do is show the submitter all other items they submitted into this table that are still active. Attached is my example application.

var rest = Ext.CreateAppRecord(Ext.TableId("TS_RESTDATASOURCE"));
rest.Read("DataServiceModScriptQuery1RESTDataSource");

var result = "";
if ( !rest.Get( result, [Pair("submitter", Shell.User().GetId()), Pair("active",0)] ) ) {
	// write an error to Event Viewer
	Ext.LogErrorMsg("Rest call failed in script " + __FILE__ + ":\n" + Shell.GetLastErrorMessage() );

	Shell.RedoMessage() = "Rest call failed";
}
else {
	var resultObj = result.from_json();
	Ext.WriteStream( "" );
}

 

The Form

On the form, I added an HTML/JavaScript Widget, with the following contents. It simply iterates the JavaScript variable injected by the ModScript and writes the values as an unordered list.

<script type="text/javascript">
document.write("<ul>");
 
for ( var i in itemsISubmittedThatAreStillOpen ) \{
    document.write("<li>");  
    document.write(itemsISubmittedThatAreStillOpen[i].Name + ": Submitted on " + new Date( parseInt( itemsISubmittedThatAreStillOpen[i].SubmitDate.match(/\d+/)[0] )));
    document.write("</li>");
}
 
document.write("</ul>");
</script>

Is my example a little contrived? Sure, the form could have invoked Data Service directly, but we are pretending that the ModScript did something important with the JSON before writing it to the form. Also, in many use cases, there is no form involved. Instead, ModScript will invoke a REST API to write data to some integration, or use ModScript to pull data from an external source into a field on the item. Also, ModScript could have written a more complex JavaScript into the page to do all the stuff we do on the form. However, writing JavaScript from inside ModScript is not very fun, as you have to be sure to encode everything correctly. Quoted text can be really annoying. Instead, use ModScript to write the data onto the form can be pretty simple, and then the JavaScript can take it from there. In a more complex example, I would move the JavaScript to its own file, simply exposing a simple function for the HTML/JavaScript widget to invoke (this makes the JavaScript reusable, easier to write using Composer's syntax highlighter, and easier to get to).

Contrived or not, we can see ModScript in action, directly invoking a REST call and doing something with the data.

 

 What's new in SBM 11.4

ModScript in 11.3.1 has the ability to make REST calls. In 11.4, we greatly increased the flexibility of the REST call functionality. Features added in SBM 11.4 for REST callouts:

  • Custom URL path parameters
    • 11.3.1 allowed the scripter to add and change URL parameters (values after the ? in the URL). 11.4 extends this to the URL path (values after the protocol, server, and port but before the ?). This allows the scripter to add a single REST Data Source that points to the protocol/server/port to whatever REST API you are interacting with, then add URL path values to invoke the REST functions required.
  • 11.3.1 allowed the scripter to invoke REST calls via POST and GET, but 11.4 extends this to PUT, DELETE, and any custom HTTP REST verb desired.
  • 11.4 allows the scripter to add any custom header to the HTTP call.
  • 11.4 allows the scripter to get the headers from the HTTP result.
  • 11.4 allows the scripter to bypass the SBM Proxy if desired.
    • SBM Proxy adds functionality such as the ability to support OATH2, but sometimes it might be desired to bypass the SBM Proxy to avoid any complexity added.
    • Bypassing can help with debugging a REST call that is having a problem (see if the problem is due to SBM Proxy or not).
  • As a side note, not related to REST but tangential to this example: In 11.3.1, ModScript has functions like Db.ReadIntegersWithSQL(), but the column types are rigid (you cannot read an int, 2 strings, and a double), which might make using Data Service look good for custom queries on the SBM database. 11.4 has Db.ReadDynaSQL(), allowing any number of columns to be returned in a query, making for extremely flexible database querying of the SBM AE schema. As such, you would only need Data Service for querying external databases. 

 

SBM ModScript - Table of Contents

Recent Comments
Oliver Kraus
Don, looks like the link to the next part is missing in this part, Oliver
Monday, 11 June 2018 8:08 AM
Don Inghram
Thanks, Oliver. All posts link back to the ModScript Blog Series table of contents.
Monday, 11 June 2018 1:01 PM
Continue reading
1815 Hits
2 Comments

SBM ModScript, Part 7 - REST Call Into ModScript

Sometimes, we want more information on a custom form, but we can't figure out how to get it. The answer might be a call to SBM JSON API using a REST widget. However, if you just can't seem to find a non-scripty way to get the data you want, consider invoking ModScript from the form. I have put together a sample application based on 11.3.1 (see 11.4 example below for updated version) which shows how you could do this. In my example, the process app has a Contact field, and I want to show more information about that contact and the company that they are part of. To do this, I wrote a ModScript that can be invoked via the Direct URL context. It requires that the contact ID be passed in, either as a URL parameter or as a JSON value in the body of the HTTP call. You wouldn't really need to provide flexibility like that, but here we are trying to give an example of both so that we can really see how to send data to ModScript in the Direct URL context.

 

The ModScript Script
var contactID = 0;
var method = "";
if ( !Shell.PostData().to_string().trim().empty() ){
	contactID = Shell.PostData().to_string().from_json()["contact"];
	method = "JSON";
}
else {
	contactID = Shell.Params()["contact"].to_string().to_int();
	method = "URLParam";
}

Here, you see that we first look at the Shell's "PostData()" value. This will hold the body of the HTTP request, as long as the request is a POST, the HTTP request "Content-Type" is "application/json", and the data length is less than our maximum allowed in the "ScriptPostDataMax" system setting (which defaults to 10 MB). So, first, the ModScript checks to see if we have any data in Shell.PostData(). If so, it casts the value to a string (most values in the Shell are Variant), then invoke from_json() on the string. Assuming that the JSON passed in was { "contact": 123 }, this should give us a Map with a single entry of "contact". So, the script immediately requests the value "contact" from the Map. Of course, the data sent in to ModScript could be quite complex, in which case you could catch the return value of "from_json()" and then process it.

If we did not get anything in Shell.PostData(), we then look in the Shell.Params(), which is a Dictionary of the URL parameters passed to this Direct URL call. In this case, we expect to find a value "contact" as a URL parameter. Values in Dictionaries are also Variant, so we need to cast it. Unfortunately, until 11.4, we do not have a direct Variant.to_int(), so we use Variant.to_string(), then use string.to_int() to finally get an integer.

Next, we want to read the Contact using the ID passed in:

var contactRec = Ext.CreateVarRecord( Ext.TableId("TS_CONTACTS") );
if ( contactID == 0 || !contactRec.Read( contactID ) ) {
	Ext.WriteStream("{}");
}

This is pretty straight forward. Create a VarRecord object for interacting with the "TS_CONTACTS" table. If we were not passed a contactID that we can read, we return an empty JSON object (the JavaScript consumer of this output expects JSON, so be sure to send valid JSON, even when there is an error).

else {
	var companyName = "";
	var companyAddress = "";
	var companyFld = contactRec.Fields().FindSysField( 200 ); // 200 is the syscode for the Contact's Company field.
	var companyID = Variant();
	if ( !companyFld.is_var_null() && fun( f, v ){ f.GetDbValue(v); return v; }( companyFld, companyID ) != 0 ) {
		var temp = Variant();
		companyFld.GetDisplayValue( temp );
		companyName = temp.to_string();
		var companyRec = Ext.CreateVarRecord( Ext.TableId( "TS_COMPANIES" ) );
		companyRec.Read( companyID );
		companyAddress = (companyRec.GetFieldValue("ADDRESS1") + ", " + companyRec.GetFieldValue("CITY") + ", " + companyRec.GetFieldValue("STATE") + " " + companyRec.GetFieldValue("ZIPCODE")).to_string();
	}

	Ext.WriteStream( [
                       "name" : contactRec.GetName(), 
                       "email" : contactRec.GetFieldValue("EMAIL").to_string(),
                       "company" : companyName,
                       "companyAddress" : companyAddress,
                       "method" : method
                      ].to_json() );
}

Here, we see a few important things. First, after calling VarFieldList.FindField() or VarFieldList.FindSysField(), you may have a null object (if the field could not be found). It is important to test for null using "is_var_null()". Next, you see a lambda which takes a Field and a Variant, it invokes Field.GetDbValue() and returns the value. This is because, until 11.4, ModScript did not have a version of Field.GetDbValue() that returned the value directly to the caller. Instead, I need a way to call Field.GetDbValue() after the check for is_var_null() but while still inside the "if" (or I could do nested "if" statements, I chose the lambda). Immediately after the lambda is declared, we invoke it with our Field and Variant, and let it give us the value that we can test.

Finally, we have the Company ID and we can read the Company to get the address. I put all of this together into a ModScript Map, then invoke "to_json()" to get a nice JSON string that JSON-encodes the embedded text values and formats the return value.

 

The JavaScript Script

On the JavaScript side, I have two examples to show both a POST with a JSON body and a GET with the data on the URL. Both put the results onto the custom form in the

location specified. The JavaScript writes an unordered list of information using the returned JSON. Keep in mind that in SBM, the jQuery object is called "jQuerySBM", not "$".

 
function processModScriptResult( data, updateLoc ) {
    console.log(data);
    var p = JSON.parse(data);
    $("#"+updateLoc).html("<ul><li>" + p.name + "</li><li><a href=\"mailto:" + p.email + "\">" + p.email + "</a></li><li>" + p.company + "</li><li>" + p.companyAddress + "</li></ul>");
}

function invokeModScriptBody( val, updateLoc ) {
    $.ajax({
        contentType: 'application/json',
        accept: 'application/json',
        type: 'POST',
        data: JSON.stringify({ "contact": val }),
        url: "tmtrack.dll?ScriptPage&ScriptName=ModScriptDirectURL",
        success: function(data){ processModScriptResult(data, updateLoc); }
    });
}
 
function invokeModScriptParam( val, updateLoc ) {
    $.ajax({
        contentType: 'application/json',
        accept: 'application/json',
        type: 'GET',
        url: "tmtrack.dll?ScriptPage&ScriptName=ModScriptDirectURL&contact=" + val,
        success: function(data){ processModScriptResult(data, updateLoc); }
    });
}

 

The Form

To pull it all together, I created a State form for my one and only workflow state. In the form's properties, I selected my custom JavaScript. I also ensured that "Include jQuery plugin" was checked. I added an HTML/Javascript widget, with the following HTML and JavaScript:

<script type="text/javascript">
  invokeModScriptParam( {Contact[ID]}, "ModScriptInsertLocation1" );
  invokeModScriptBody( {Contact[ID]}, "ModScriptInsertLocation2" );
</script>
<p id="ModScriptInsertLocation1"></p>
<p id="ModScriptInsertLocation2"></p>

This invokes my two examples, and injects a couple locations in the HTML for the JavaScript to write out the results.

 

What's New in 11.4

I wrote this application to work in 11.3.1. In 11.4, we could have simplified our script. Also, we could bind the JSON output directly to a REST Grid Widget on a form. I  have updated the sample application with the changes. A few notable differences:

  • Ext.SetContentType( "application/json" );
    • In 11.4, ModScript can set the declared Content-Type for the Direct-URL context. This is important as the REST Grid Widget does not allow you to bind to a request that returns a Content-Type of "text/html".
  • The JSON returned is now a JSON array so that the REST Grid Widget recognizes it.
  • Since we are using the REST Grid, we don't really need the HTML/JavaScript widget or the JavaScript file. However, I kept them so we can see all the different options. The JavaScript doesn't need to call JSON.parse(data), the new Content-Type of "application/JSON" indicates to the underlying engine that it should parse the JSON and give us an object. The script was changed to access index 0 of the array, as the JSON is now an array with our object inside.
  • To use the REST Grid Widget, I added an Endpoint that points to the ModScript call that we had been making via JavaScript. Be sure to edit the endpoint in Application Repository to point to your AE server. The REST Grid Widget can bind the Contact ID to the URL parameter.
  • Other cleanup in ModScript:
    • Invoke "Variant.to_int()" directly
    • Use "VarRecord.GetDbValueInt()" to get the field value as an integer
    • Setting our companyName string = Variant directly (no longer need to invoke Variant.to_string() when assigning a Variant value to a string).
    • 11.4 has VarRecord.GetFieldValueString(), but it still requires the string to be passed in rather than returned directly to the caller (this is so that it can return false if the field is not found). I added a function to the Field class in my script that will return the string value directly to the caller, which makes it easier to concatenate the address string. Only do this if you are SURE all the fields will be found:
      • def VarRecord::GetFieldValueString( s ) {
        	var ret = "";
        	this.GetFieldValueString( s, ret );
        	return ret;
        }

 

SBM ModScript - Table of Contents

Recent comment in this post
Oliver Kraus
Good example (although I prefer plain JS), I did not know that Shell has a PostData() member function. Good to know.Thanks, Olive... Read More
Monday, 11 June 2018 7:07 AM
Continue reading
1855 Hits
1 Comment

SBM ModScript, Part 6 - Invoking DLLs

ModScript has the ability to invoke a function exposed from a DLL. The parameters passed to the DLL from ModScript are input/output, meaning that ModScript can send any data to the DLL, and the DLL can send any data back. However, the DLL function must have a specific function signature, so it is not possible to invoke a DLL function that was not designed to be called from ModScript.

 

The DLL

I'll use C++ for the DLL. First, declare the type "SBMScriptArg". This will be they type for each function parameter passed from ModScript to the DLL:

struct SBMScriptArg
{
  char* pData;
  int   size;
};

Also, typedef the ReallocArg_t function, which is a callback function that the DLL can use to resize a parameter, ensuring it will be big enough for the output value:

typedef int( *ReallocArg_t )( SBMScriptArg* pArg, int newSize );

Finally, give yourself a set of functions for setting an argument to a string, int, etc:

void setArg( const char* s, size_t len, SBMScriptArg& arg, ReallocArg_t reallocArg )
{
  assert( len < INT_MAX );
  reallocArg( &arg, static_cast( len + 1 ) ); // checks if arg size is already big enough, we don't need to also check
  memcpy( arg.pData, s, len );
  arg.pData[len] = 0;
}

void setArg( const char* s, SBMScriptArg& arg, ReallocArg_t r )
{
  setArg( s, s ? strlen( s ) : 0, arg, r );
}

void setArg( const std::string& s, SBMScriptArg& arg, ReallocArg_t r )
{
  setArg( s.c_str(), s.size(), arg, r );
}

template 
inline void setArgT(  T n, SBMScriptArg& arg, ReallocArg_t r )
{
  setArg( std::to_string( n ), arg, r );
}

With this in place, the DLL can reset any parameter sent from ModScript to a new value, be it a string with SetArg(), or an int, double, etc with SetArgT(). Now, define the function that ModScript will invoke. In my example, the function is called "DoIt".

extern "C"
__declspec( dllexport ) int DoIt( SBMScriptArg* args, int argCount, ReallocArg_t reallocate )
{
  for ( int ii = 0; ii < argCount; ++ii )
  {
    printf( "DLL: %s\n", args[ii].pData );
  }

  if ( argCount > 0 )
  {
    setArg( "what", args[0], reallocate );
  }

  if ( argCount > 1 )
  {
    setArgT( 12345, args[1], reallocate );
  }

  return 5;
}

The extern "C" __declspec( dllexport ) ensures that ModScript will be able to find the function in the DLL. The function must return an int, and have the parameter signature ( SBMScriptArg* args, int argCount, ReallocArg_t reallocate ). After that, the function can do whatever you desire. In my use case, the function first loops through the parameters passed in and prints the value to console (may not be useful if you are running ModScript in AE under IIS). Then, if there is at least 1 parameter, it sets the first parameter to "what". If there are at least 2 parameters, it sets the second to 12345. These values are visible in ModScript after the DLL function completes. Finally, an integer value is returned. I have bundled this C++ together in an example Visual Studio project.

Notes:

  • Be sure to save the source code for the DLL, possibly in a zip that you include on the SBM Application Engine machine, right next to the DLL.
  • It is usually best to compile in "release" mode, as "debug" mode often requires debug C++ runtime DLL files that are only present on machines that have Visual Studio on them.
  • Always compile 64 bit binaries.
  • The DLL must be on all AE runtime machines, both for User Workspace and Web services components.
  • ModScript can load a DLL from an absolute path, but it is usually better to place the DLL in the folder that is specified in the "ScriptAppPath" entry in the Windows Registry. This topic is covered in more detail in the Composer help topic "Loading the Library in SBM ModScript".

 

The Script
var lib = CreateObject("SBMLibrary");
lib.SetLibraryName("ScriptDllTest");
lib.LoadLibrary();
var param1 = Variant("abc");
var param2 = Variant("def");
var ret = lib.CallLibraryFunction( "DoIt", param1, param2 );
Ext.WriteStream( "${ret} : ${param1} : ${param2}" );

The output is "5 : what : 12345". As you can see, ModScript passed two parameters to the DLL, the DLL can use those values to do whatever it needs to do, and it can also change those values to create output values that ModScript can use. As the parameters can be output, it is important that they are created before the call to "Lib.CallLibraryFunction()"; creating them inline creates const-temporary values that cannot serve as output.

 

Improvements in 11.4

I am not a big fan of Variant. I added it to ModScript in order to make it possible to convert AppScripts to ModScripts. However, I accidentally made it far more important than I intended, as many functions require that the Variant be created before being passed in, and many others have Variant as the return type. In 11.4, I created optional function signatures that do not require Variant. In this case, the Lib.CallLibraryFunction() now has an optional signature that takes string& for the parameters rather than Variant&. The documentation in 11.4 is much improved, adding the missing "Lib" class which is the underpinning class for this functionality.

 

SBM ModScript Table of Contents

Continue reading
1544 Hits
0 Comments

SBM ModScript, Part 5 - Algorithms and Lambdas

ModScript uses the ChaiScript engine, which supports lambdas and has many algorithms built into it. These tend to go hand-in-hand. Lambdas are functions that do not have names. They can be passed like function pointers as parameters to other functions, or they can be directly invoked if desired. Algorithms often take a function and execute the function against a list of items, so lambdas are often useful in this context. They keep the logic you are invoking right in front of you, rather than requiring you to track down the function in order to understand what is happening. The great thing about using algorithms is that you don't have to write all the logic yourself.

 

In Part 2 of this series, I used the any_of() algorithm, along with a lambda, to check all records in an AppRecordList to see if any were still active. For reference:

if ( !any_of( itemIDs,
		fun( itemID ) { // return true if item is active
			var contained = Ext.CreateProjectBasedRecord( CONTAINER_TBL );
			return contained.Read(itemID) &&
				contained.GetFieldValue( "ACTIVEINACTIVE" ) == 0;
		} ) ){
	...
}

Without an algorithm, I'd need to:

  • Create a variable called "found" to flag if I found an active item
  • Use a for loop to iterate the items in the list
  • Check each record for active
  • Set the "found" variable if I found an active item
  • break
  • Check the flag using an if statement

With the algorithm, I can fit all that code inside my if statement. If you don't like my lambda, you could have defined the function elsewhere and used the function name (function names are function pointers, they can be passed as parameters to functions).

 

Ranges in ChaiScript

Before I can talk about algorithms, I need to discuss what a range is. A range is a tool for iterating a container. It consists of two iterators, one that points to the front of the container, and one that points to the back of the container. The idea is that you can move through the range forwards by popping the front or move through the range backwards by popping the back, all without changing the number of items in the underlying container. Vectors, Maps, strings, AppRecordLists, and Dictionaries all can be used as a "range" for iteration. The following functions can be invoked on a range:

  • bool empty()
    • Returns true if the range has no items in it.
  • void pop_front()
    • Moves the front iterator forward. The underlying container is not changed. If the range is empty, this throws an exception.
  • void pop_back()
    • Moves the back iterator backward. The underlying container is not changed. If the range is empty, this throws an exception.
  • front() : return type is the type of the item at the front of the container
    • Returns the item that the front iterator is pointing at. If the range is empty, this throws an exception.
  • back() : return type is the type of the item at the back of the container
    • Returns the item that the back iterator is pointing at. If the range is empty, this throws an exception.

 

Algorithms Available in ChaiScript
  • void for_each( container, func )
    • Iterates a range, executing a function on each item.
  • bool any_of( container, func )
    • Iterates a range, returns true if the function passed in returns true for any item in the range. "none_of()" could be created by simply invoking !any_of().
  • bool all_of( container, func )
    • Iterates a range, returns true if the function passed in returns true for all items in the range.
  • range find( container, valueToFind )
    • Returns a range with the front iterator pointing to the item found, or an empty range if the item was not found.
  • range find( container, valueToFind, compare_func )
    • Invokes compare_func on each item in the container, passing "valueToFind" and the current container entry; returns a range with the front iterator pointing to the first item for which compare_func returns true.
  • bool contains( container, valueToFind )
    • Returns true if any item in the container is equal to "valueToFind".
  • bool contains( container, valueToFind, compare_func )
    • Invokes compare_func on each item in the container, passing "valueToFind" and the current container entry; returns true if compare_func returns true.
  • foldl( container, func, initial ) : return type is the type of the "initial" parameter
    • Performs the function "func" over the container. Starts with "initial" and continues with each element. For example, "sum" invokes the `+` function on each item, starting with a value of 0.0 and returning the sum of items in the range.
  • double sum( container )
    • Returns the sum of items in the container. Items must be able to be added.
  • double product( container )
    • Returns the product of items in the container. Items must be able to be multiplied.
  • concat( containerA, containerB ) : return type is the type of the "containerA" parameter
    • Returns a new container containing the contents of containerA followed by the contents of containerB. AppRecordList cannot be used with this algorithm.
  • take( container, num ) : return type is the type of the "container" parameter
    • Returns a new container containing up to "num" elements copied from "container". AppRecordList cannot be used with this algorithm.
  • void take( container, num, inserterFunc )
    • Invokes "inserterFunc" with up to "num" elements from "container". For instance, the previous "take()" signature, which returns a new container, invokes this "take()" signature with the "back_inserter()" function which wraps the new container it will return; "back_inserter()" calls "push_back()" on the new container for each item passed to it. AppRecordList can be used with this algorithm as input, but "inserterFunc" should be something like back_inserter() wrapped around a Vector.
  • take_while( container, func ) : return type is the type of the "container" parameter
    • Returns a new container containing the first "n" elements copied from "container" for which "func()" returns true. AppRecordList cannot be used with this algorithm.
  • void take_while( container, func, inserterFunc )
    • Iterates container and invokes "func()" on each item until "func()" returns false. For the first "n" items for which "func()" returns true, invokes "inserterFunc()". For instance, the previous "take_while()" signature, which returns a new container, invokes this "take_while()" signature with the "back_inserter()" function which wraps the new container it will return; "back_inserter()" calls "push_back()" on the new container for each item passed to it. AppRecordList can be used with this algorithm as input, but "inserterFunc" should be something like back_inserter() wrapped around a Vector.
  • drop( container, num ) : return type is the type of the "container" parameter
    • Returns a new container with values from "container" copied to it, skipping the first "num" items. AppRecordList cannot be used with this algorithm.
  • void drop( container, num, inserterFunc )
    • Invokes "inserterFunc()" with values from "container", skipping the first "num" items. AppRecordList can be used with this algorithm as input, but "inserterFunc" should be something like back_inserter() wrapped around a Vector.
  • drop_while( container, func ) : return type is the type of the "container" parameter
    • Iterates container and invokes "func()" on each item until "func()" returns false. Returns a new container with all items copied from "container" starting where "func()" returned false. For example, "string::ltrim()" uses this to skip past white spaces in the string, copying the rest of the string into the return value; In Part 2 of this series, I wrote a custom trim function which trims commas off of the front and back of a string using "drop_while()". AppRecordList cannot be used with this algorithm.
  • void drop_while( container, func, inserterFunc )
    • Iterates container and invokes "func()" on each item until "func()" returns false. Invokes "inserterFunc()" with values from "container" starting where "func()" returned false. AppRecordList can be used with this algorithm as input, but "inserterFunc" should be something like back_inserter() wrapped around a Vector.
  • reduce( container, func ) : return type is the type of the first item in the "container" parameter
    • The "container" parameter must have at least two items in it. Initializes return value with the first item in "container", then iterates the remaining elements in container, invoking "func()" by passing in the current return value and the current element.
  • string join( container, delim )
    • Iterates container, invokes "to_string()" on each element and appends to the result string, with each entry separated by "delim".
  • filter( container, func ) : return type is the type of the "container" parameter
    • Returns a new container with values copied from "container" for which "func()" returns true. AppRecordList cannot be used with this algorithm.
  • void filter( container, func, inserterFunc )
    • Iterates container, invokes "inserterFunc()" on each item for which "func()" returns true. AppRecordList can be used with this algorithm as input, but "inserterFunc" should be something like back_inserter() wrapped around a Vector.
  • Vector generate_range( iterator1, iterator2 )
    • For each item between "iterator1" and "iterator2" (inclusive), copy the values into the resulting Vector.
  • Vector generate_range( num1, num2 )
    • Creates a Vector with numeric values between "num1" and "num2", inclusive.
  • void generate_range( iterator1, iterator2, inserterFunc )
    • For each item between "iterator1" and "iterator2" (inclusive), invoke "inserterFunc()". AppRecordList can be used with this algorithm as input, but "inserterFunc" should be something like back_inserter() wrapped around a Vector.
  • void generate_range( num1, num2 , inserterFunc )
    • For each number value between "num1" and "num2" (inclusive), invoke "inserterFunc()".
  • Vector zip( collection1, collection2 )
    • Iterates collection1 and collection2 at the same time, returns a Vector where each entry is a Vector with the corresponding elements from collection1 and collection2. The resulting Vector will have the same number of elements as the smaller of the two collections. Example: zip( [1,2], [3,4,5] ) returns [ [1,3], [2,4] ].
  • Vector zip_with( func, collection1, collection2 )
    • Iterates collection1 and collection2 at the same time, returns a Vector where each entry is the return value from executing "func()" with the two corresponding elements from collection1 and collection2. The resulting Vector will have the same number of elements as the smaller of the two collections. Example: zip_with( `==`, [1,4,5,6], [0,4,5] ) returns [ false, true, true ].
  • void zip_with( func, collection1, collection2, inserterFunc )
    • Iterates collection1 and collection2 at the same time, invokes "func()" passing the corresponding elements from collection1 and collection2, passing the return value to "inserterFunc()" (inserterFunc() is often a "back_inserter()" on a Vector. For instance, the previous "zip_with()" signature creates a Vector to return, then invokes this "zip_with()" function, passing "back_inserter(myRetVect)" as the "inserterFunc".
  • reverse( container ) : return type is the type of the "container" parameter
    • Reverse the order of items in the container, returning a new container to the caller. AppRecordList cannot be used with this algorithm.
  • function bind( func, param1-n )
    • Creates an invoke-able function which can save some parameters passed at bind-time, along with some parameters passed at invoke-time, to the underlying function. Any parameter that is an underscore "_" implies that the resulting function will take a parameter, and the parameter passed to that function will be passed to the bound function.
    • For example, the implementation of "back_inserter( container )" is bind(push_back, container, _ );, meaning that the returned function takes a single parameter (just one underscore), when the returned function is invoked. Each call to the function returned from "back_inserter( container ) " will invoke push_back( container, XXX ), where XXX is the value passed to that function. FYI, in ChaiScript, a class method is the same as a function that takes an object of that class as the first parameter; as such, for a Vector v, there is no difference between v.push_back( x ) and push_back( v, x ).
    • Example: 
      var b = bind( fun( a, b, c ) { return [a,b,c]; }, _, 3, _ ); 
      b( 1, 2 );
      Step by step: using a lambda that takes 3 parameters and creates a Vector out of them, "bind" will create a function that takes 2 parameters (see 2 underscores), and passes param1, the literal 3, and param2, to the lambda. Now, "b" will be a function that takes 2 arguments. We then invoke "b" with values 1 and 2, which results in a Vector of [ 1, 3, 2 ]. The "bind()" function can be very helpful when building a function pointer to pass to an algorithm.
  • function back_inserter( container )
    • Returns a function that takes a single parameter. Any time the function is invoked, the value passed as the parameter will be forwarded to a call to "container.push_back()".
  • min( v1, v2 ) : return type is the type of the "v1" parameter or the "v2" parameter, whichever is lower
    • If v1 < v2, v1 is returned; otherwise v2 is returned.
  • max( v1, v2 ) : return type is the type of the "v1" parameter or the "v2" parameter, whichever is higher
    • If v1 > v2, v1 is returned; otherwise v2 is returned.

 

SBM ModScript Table of Contents

Recent Comments
Paul Thompson
Don: Why do I get an error calling "DoStuff()" and is there anyway to get "pbr" into the context of the lambda? Maybe a Singleton... Read More
Friday, 04 October 2019 4:04 AM
Don Inghram
Hi, Paul. In ChaiScript, lambdas cannot access items that are not passed directly to the lambda. In this case, pbr is not a variab... Read More
Wednesday, 09 October 2019 10:10 PM
Paul Thompson
Don: Don't take this he wrong way :-)This article wins the prize for weight and "density", as in "a lot of information packed into... Read More
Wednesday, 27 March 2019 4:04 AM
Continue reading
1673 Hits
3 Comments

SBM ModScript, Part 4 - JSON

One useful feature of the ChaiScript engine is the support for JSON. As ModScript can be invoked directly via the Direct URL context, a JSON body could be sent by the caller and ModScript can parse it and then use it. Also, as ModScript can make REST call-outs, the fact that ChaiScript can format JSON for us empowers us to really get things done.

JSON is a convenient format for passing data in a web application as it is compact and well defined. Here is an example:

{
	"arrayOfInts": [1,2,3,4,5],
	"intData": 5,
	"doubleData": 4.5,
	"stringData": "Welcome to ModScript"
}
 
Parsing JSON

In the above JSON sample, the curly braces { ... } indicate the beginning of an object, with name-value pairs. The square braces [ ... ] indicate an array. When the ChaiScript engine parses a JSON string, integers become ints, floating points become doubles, text becomes strings. JSON arrays become Vectors. JSON objects become Maps which can store name-value pairs; it is important that the JSON not have repeating name entries in the same object as Maps can only store unique key values. With this in mind, the following is an ModScript excerpt that will parse the JSON from a string (as the JSON text has embedded quotes, we escape them with backslashes). The following script will write "Welcome to ModScript" as an Information entry in the Application Event Log:

Ext.LogInfoMsg(
"{
	\"arrayOfInts\": [1,2,3,4,5],
	\"intData\": 5,
	\"doubleData\": 4.5,
	\"stringData\": \"Welcome to ModScript\"
}".from_json()["stringData"] );

Step by step:

  • We see a function call to Ext.LogInfoMsg(), which will take the output of the inner code and write it to the Application Event Log.
  • Inside, we see a string literal with the JSON embedded in it, with quotes escaped with backslashes. (Yes, string literals can have embedded newlines in them. The resulting string will have the newlines in the string).
  • After the string literal, we see a direct call to from_json(), which parses the JSON string into a Map (because the outer most part of the JSON is a JSON object).
    • The Map has the following keys: "arrayOfInts", "intData", "doubleData", "stringData"
    • The "arrayOfInts" entry in the Map is a Vector, and each entry in the Vector is an int.
    • The "intData" entry in the Map is an int, "doubleData" is a double, "stringData" is a string.
  • Finally, we see ["stringData"], which calls the lookup operator on the Map, returning the corresponding entry. In this case, it returns the string "Welcome to ModScript", which is what gets sent to the Ext.LogInfoMsg() function call.

Notes:

  • JSON objects can have entries that are objects. In this case, after a call to from_json() you will get a Map with a key-value entry where the value is a Map.
  • Entries in a JSON array do not all need to be the same type. A JSON array could have a text entry followed by an integer entry followed by an object entry. In this case, after a call to from_json() you will get a Vector with a string, an int, and then a Map. As such, the from_json() function can parse any JSON string, as long as the JSON objects do not have duplicate key names in them.
  • The top level of the JSON does not need to be an object, it can be an array, a string, an integer, or a double.
    • The return value of from_json() may be a Map, Vector, string, int, or double.
    • You may have noticed in Part 2 that we used from_json() to take a string with comma-separated integers and turn it into a Vector. When working with Multi-Relational, Multi-Selection, or Multi-Group fields, getting the internal value will return a comma-separated list of integers with commas at the beginning and end (example: ",65,732,899,"). Trim these commas, append square braces, and you have a JSON array (example: "[65,732,899]") which from_json() can parse into a Vector of integers.

 

Formatting JSON

The ChaiScript Engine also provides a to_json() function. This can be invoked on a Map, Vector, string, int, or double, and will generate a JSON string from the object. This is very helpful, especially when the scripter is pulling text from various locations, as they do not need to worry about encoding the quotes, etc, while building the JSON string. In a future blog article about using ModScript to invoke REST calls, you will see the creation of the HTTP message body that looks like this:

["fixedFields":false, "fields":[["dbname":"TITLE"], ["dbname":"STATE"]]].to_json()

What you see here is an inline ChaiScript Map (name value pairs created with a : separator). Inside the Map is a "field" entry which is a Vector of Maps. The resulting JSON string:

{
  "fields" : [{
      "dbname" : "TITLE"
    }, {
      "dbname" : "STATE"
    }],
  "fixedFields" : false
}
 
Formatting JSON From Variant

The to_json() utility does not play nicely with Variant. This is mainly because it is not clear what JSON type the Variant should be mapped to: is it a string, an int, a string that contains an int? Due to this ambiguity, there is no easy way to make Variant play nicely with to_json(). However, Variant has a member method Variant.to_string(). With this, you can get a string from the Variant, and string has other conversion methods like string.to_int(), so chaining a Variant "v" as such: v.to_string().to_int() will give you the value of the Variant as an int. SBM 11.4 introduced many methods to avoid Variant in general, and it also added functions like Variant.to_int().

// an example for 11.3.1
Ext.LogInfoMsg(
  [ 
    Shell.Item().GetFieldValue("TITLE").to_string(), 
    Shell.Item().GetFieldValue("STATE").to_string().to_int()
  ].to_json() 
);
// equivalent example for 11.4
Ext.LogInfoMsg(
  [ 
    Shell.Item().GetFieldValueString("TITLE"), 
    Shell.Item().GetFieldValueInt("STATE") // or Shell.Item().GetFieldValue("STATE").to_int()
  ].to_json() 
);

Step by step:

  • We see a function call to Ext.LogInfoMsg(), which will take the output of the inner code and write it to the Application Event Log.
  • Inside, we see the beginning of an inline Vector
  • The first entry in the Vector will be the string text from the "TITLE" field.
  • The second entry in the Vector will be the integer value (internal value) of the "STATE" field.
  • The to_json() function is invoked on the Vector, creating a JSON string.
  • The output JSON is sent to the call to Ext.LogInfoMsg().
  • An example of the output: ["Test item 1", 27]

Notes:

  • to_json() does not work with Variant directly (or a Map/Vector which contains a Variant). A Variant can be converted to a string using Variant.to_string(), and the returned string can then be converted to int using string.to_int(). SBM 11.4 added functions like Variant.to_int() and Variant.to_double() so that the value does not first need to be converted to a string.
  • The output text from to_json() has newlines and white-space, which makes the JSON readable.
  • The use of a Map causes the entries to be reordered.

 

SBM ModScript Table of Contents

Continue reading
1945 Hits
0 Comments

SBM ModScript, Part 3 - Adding Methods to a Class

SBM ModScript uses the ChaiScript engine. ChaiScript is a fast, modern scripting engine that was written in C++. As it was written by a C++ developer, it has a bit of a C++ flavor to it. For instance, the growable array class is named Vector and is, in-fact, implemented using the C++ std::vector container. In some ways, this may make it easier to identify what the methods for the classes in ChaiScript will look like.

However, ChaiScript does not expose every C++ function from a given class into the scripting language. For instance, the string class does not include the string.replace() function. Nevertheless, ChaiScript DOES let you extend a class to add more functions to it. As such, we can write the "missing" function in a ModScript, then include() that script in other scripts. It is a "best practice" to build a library of helpful utilities.

Here is how I would add the string.replace() function in ModScript:

def string::replace( size_t pos, size_t len, string s ) {
	if ( pos > this.size() ) {
		this.substr( this.size()+1, string_npos ); // throws out_of_range, which is expected
	}

	var ret = this.substr( 0, pos );
	ret += s;
	if ( len != string_npos && pos + len < s.size() ) {
		ret += this.substr( pos + len, string_npos );
	}

	this = ret;
	return this;
}

 

SBM ModScript Table of Contents

Recent comment in this post
Paul Thompson
Good demo. Short, focused, to-the-point.
Wednesday, 27 June 2018 4:04 AM
Continue reading
1976 Hits
1 Comment

SBM ModScript, Part 2 - Transition Related Items

In part 1 we reviewed what ModScript is. Let's start looking at some examples:

In Use Case 1, the ModScript will run in the Post Transition context of an item's Close transition. The item might be selected in a multi-relational field on a separate item, and if all items in that multi-relational field are closed, we'll transition the container item. This is similar to what can be done with the SBM Sub-Tasks feature, but we'll do it via scripting because there could be some extra logical detail that Sub-Tasks couldn't check for but scripting could (and it gives us a showcase for ModScript features). The full application can be found here: Container.zip. This script uses the algorithms "drop_while()" and "any_of()", to see more information on algorithms, see Part 5. This script also uses "from_json()", to see more about ChaiScript's JSON utility functions, see Part 4.

 

// set up some constants for use later in the script
add_global_const( "USR_CONTAINER", "CONTAINER_TBL_NAME" );
add_global_const( "RELATED_ITEMS", "CONTAINER_FIELD" );

// Get the container application table id and add it as a global
global CONTAINER_TBL = Ext.TableId(CONTAINER_TBL_NAME);

// define a function that trims commas and returns the new string
def TrimCommas( s ) {
/*	ChaiScript engine automatically returns whatever occurs on the last line of a 
	function. We could add a "return" statement here if desired for clarity.
	This may be a little hard to follow, but we call drop_while on our string to remove 
	the starting commas, then the return value is reversed, we do drop_while again to 
	trim the back, then reverse again.
	These "fun" statements are anonymous functions known as lambdas */
	reverse( drop_while( reverse( drop_while( s, fun(x){ return x == ','; } ) ), 
		fun(x) { x == ','; })); 
}

// Find the multi-relational field
var relational = Ext.CreateAppRecord( Ext.TableId("TS_FIELDS"), 
				FieldTypeConstants.MULTIPLE_RELATIONAL );
relational.ReadByColumnAndColumn("TABLEID", CONTAINER_TBL, "DBNAME", CONTAINER_FIELD );

// Create a list, as it is possible that our item is in more than one container item
var containerList = Ext.CreateAppRecordList( CONTAINER_TBL );

/* Yes, ModScript has multi-line comments!

 Read the list of items that contain this item. 
 Use SQL binding by passing a Vector of Pair objects, each with a data type and a value.
 Vectors can be created on the fly using [ ... ] syntax */
containerList.ReadWithWhere(
	"TS_ID in (select TS_SOURCERECORDID from TS_USAGES where TS_FIELDID=? and TS_RELATEDRECORDID=?)",
	[ Pair(DBTypeConstants.INTEGER, relational.GetId()), 
	  Pair(DBTypeConstants.INTEGER, Shell.Item().GetId()) ] );

// loop through the resulting list, using the "for each" syntax (just a ":")
for( containerItem : containerList ) {
	// for each item, read the contents of the relational field, check the items, 
	// transition if necessary
	
	// Get the field value from the containing item. GetFieldValue() returns a Variant,
	// so use to_string() to get it as a string.
	// In 11.4, we have GetFieldValueString(), GetFieldValueInt(), etc, for getting field 
	// values as the desired type.
	var fieldVal = containerItem.GetFieldValue(CONTAINER_FIELD).to_string();
    
	// remove the current item from the comma-separated list of items 
	var regex = Regex();
	regex.Compile( ",${Shell.Item().GetId()}," );
	fieldVal = regex.ReplaceAll( fieldVal, "," );

	// trim outside commas off of list
	fieldVal = TrimCommas( fieldVal );
	
	// turn comma separated list into Vector of values. 
	// ChaiScript can do this with "from_json" if we use array syntax [ ... ]
	// ChaiScript has in-string processing, use ${ ... } inside a string and the stuff 
	// inside the braces will be processed by the engine and put inline in the string.
	var itemIDs = ("[${fieldVal}]").from_json();

	// Loop through those other contained items to see if they are inactive too.
	// We could do another for-each loop here, but let's use an algorithm instead.
	if ( !any_of( itemIDs, 
			fun( itemID ) { // return true if item is active
				var contained = Ext.CreateProjectBasedRecord( CONTAINER_TBL );
				return contained.Read(itemID) && 
					contained.GetFieldValue( "ACTIVEINACTIVE" ) == 0;
			} ) ){
		// no items found that are active, we need to transition this container item
		containerItem.QuickTransition( "CONTAINER.CLOSE", true );
	}
} 

 

SBM ModScript Table of Contents

Recent Comments
Don Inghram
My SQL is correct, we are looking for items, which have the Multi-Relational field specified, in which the current item is selecte... Read More
Tuesday, 26 June 2018 6:06 PM
Paul Thompson
Don: Should line 34: "TS_ID in (select TS_SOURCERECORDID from TS_USAGES where TS_FIELDID=? and TS_RELATEDRECORDID=?)" be: "TS_ID i... Read More
Sunday, 24 June 2018 4:04 AM
Curtis LaPrise
Being new to Modscript, Chaiscript, and/or C++ this was difficult to follow. I generally get the gist of it, though I couldn't de... Read More
Tuesday, 30 October 2018 3:03 PM
Continue reading
2226 Hits
3 Comments

SBM ModScript, Part 1 - An introduction to ModScript

SBM has had a scripting engine for two decades now called AppScript. It is powerful and fast. It is also based on VBScript, and was crying out for a refresh. SBM gained a new scripting engine in 11.3 named ModScript. AppScript is still supported.

So, what is ModScript? ModScript is based on the ChaiScript engine. When reading about ChaiScript, it is important to separate in your mind the documentation meant for C++ developers from the documentation about the scripting language. The C++ commentary and documentation is there for the implementers of languages like ModScript, not for you, the consumers of the scripting language. In order to get some visibility into ChaiScript, you can peruse the unit tests and samples, again, be sure to view ".chai" files, not ".hpp" or ".cpp" files. However, this series will have example scripts, so you don't necessarily need to dive too deeply into those long lists of files at the moment. For now, know that ChaiScript is a modern style script language with keywords like "for", "while", "continue", "break", "if", "else", "switch", etc, and it allows the script writer to create functions, classes, variables, global variables, global constants, function pointers, lambdas, and more. Much of the syntax is similar to JavaScript; however, the variables are strongly typed (once they are assigned a value of type "int", they cannot become a "string"). ChaiScript also has the concept of a Vector (array which can grow) and a Map (a binary search tree which stores key-value pairs, the key is a "string"). Furthermore, it has many utility functions including to_json() and from_json(), which know how to interact with Vector, Map, string, int, and double data types to create and parse valid JSON.

ModScript is more than ChaiScript; a scripting engine can't do much without an API for interacting with the host program. We added all the functionality of AppScript and more. We also created a conversion tool that converts scripts from AppScript to ModScript. As we already had a scripting engine in place, we were able to piggyback off of AppScript to create identical integration points for ModScript:

  • Pre-Transition
  • Post-Transition
  • Pre-State
  • Post-State
  • Notification
  • Self Registration
  • HTML Template
  • Direct URL
  • Database Import.
  • SOAP - In 11.4, we add a SOAP entry point, allowing ModScripts to be invoked by SOAP callers (including the Orchestration Engine and the new Scheduler).

With all these integration points, scripts can be invoked in nearly all aspects of SBM. In fact, developers who are familiar with AppScript Contexts will find that the ModScript runtime contexts are identical, providing the same information and access. All of AppScript's Ext functions are available in ModScript, such as Ext.CreateAppRecord() and Ext.TableId(). All of AppScript's classes are available, such as AppRecord, VarRecord, ProjectBasedRecord, and User, and all those classes have the same (or more) methods available in ModScript.

What we added to ModScript, above and beyond the functionality we copied from AppScript, is what I am very excited about:

  • Time zone awareness
  • Locale awareness
  • Date/time string formatting and parsing using time zone and locale rules
  • Regular expressions
  • REST call-outs (improved in 11.4)
  • Logging to files
  • Logging to Active Diagnostics (11.4)
  • ModScript can execute transitions on items from primary tables (AppScript could only Update, not Transition).
  • ModScript is available in a limited, read-only capacity in SaaS, as it is written from the ground up to be namespace safe.

Not only is ModScript feature rich, the features are growing. Almost every question from early adopters has lead to an enhancement to ModScript. For instance, a question about how to generate a temporary file name using the current time led the the creation of the TimeMillis class, which can interact with time down to the millisecond, as will as the TempFile class, which creates a temporary file and provides the file path name (the temp file gets deleted when the TempFile object goes out of scope). These enhancements are coming in 11.4, along with the ability to format and parse date/time strings using custom formats and more.

As an example of the features available in ModScript, consider REST interaction. ModScript can handle direct call-ins via the direct URL usage, and the script has the HTTP body available to it, making it possible to POST data to ModScript as JSON and have ModScript parse it, act on it, and write JSON back to the caller. In addition to that, ModScript can invoke REST calls using GET and POST, with all other HTTP verbs available in 11.4. ModScript gives you the ability to publish a single REST endpoint from Composer, which can be customized per-runtime in AR, and ModScript can modify the URL path and URL parameters so that one endpoint can serve any number of REST entry points from a REST API. It can send custom headers and see the HTTP headers from the response (11.4). As it can use the SBM Proxy, ModScript call-outs can take advantage of the OATH2 token exchange in order to fully support OATH2 end points. Opening up REST in this way completely opens the doors of what is possible.

From the hints above, it is clear that ModScript is still growing. Many features were added in 11.4 to extend functionality, improve usability, and fill gaps. The documentation coming in 11.4 is an overhaul of what was available previously, making the functionality much more understandable. We included many more examples, and corrected bad examples, to help get you going. Also in 11.4, the Composer Validate button becomes useful again, checking for missing variable/function declaration (which helps catch misspelled variable/function names). We are dedicated to helping you be successful with scripting and removing any frustration from the SBM experience.

 

SBM ModScript Table of Contents

Continue reading
2087 Hits
0 Comments

Recent Tweets