Basic Concepts

To understand how to work with the Core, it is necessary to accept several basic concepts:

The Concept of “Class”

The concept of “Class” in the Core refers to a JavaScript class inherited from a base class that provides loading of profiles for the corresponding entity in the database, connection to the database based on profiles, and other useful methods. Hereafter, the term “Entity” may be encountered, which implies the class, its profiles, methods, data in the database, forms, and tables on the frontend side.

Class Profile and Client Object Profile

A profile contains settings for the class itself and each of its fields. One of the most important, but far from the only, tasks of the profile is to provide the Core with information on how to build SQL queries to the database, including relationships between tables (see the point below), various default parameters such as sorting, limits, and applying restrictive conditions.

Unlike classic ORMs, the system for describing entity fields of a class allows for the description of fields not only from the main table but also fields from other entities that will be pulled in by the system via JOINs. Thus, a profile creates something similar to a VIEW in classic relational DBMSs, but of course, the description of fields in a profile is not limited to this and serves many other purposes.

This is a basic concept. When a developer designs entities for a solution, fields from related tables are incorporated into the class from the start. For example, in the user class, there is not only a country_id field but also country_name and country_code, which are virtual, meaning they are pulled from another table (ds_user_country) by the country_id key. These relationships are embedded in the profile, not built by the developer during the query. Of course, during a query, the user can limit the set of fields they need, and unnecessary JOINs will not be assembled.

The initial creation of the profile and fields is done in the tables.json file, after which it can be modified through the system interface. In tables.json, only the most essential information is described, as most settings have optimal default values.

Automatically Created Fields

All tables automatically have system fields such as created (datetime), updated (datetime), deleted (datetime) (the Core operates on the principle of soft deletion), created_by_user_id, last_edited_by_user_id, deleted_by_user_id, and their virtual fields for pulling in names, as well as some other system fields for various mechanisms. Additionally, for tables with hierarchical relationships, fields like node_deep, parents_count, and children_count are automatically created, and their content correctness is automatically maintained by the Core. The count of parents and children in these fields indicates the total count across all depths, not just one level (direct descendant/parent).

In addition to query building functions and other database interactions (e.g., whether records can be created in this table, or setting values for specific fields during creation or editing), the profile contains various settings, most of which are used on the frontend to determine the appearance and functionality of components. There are settings for the entire class and settings for each field. Examples of class settings include the number of records per page (for building pagination in tables), the name of the form to open from the table’s context menu, and the title. Examples of field settings include the Name, tooltip text, editor type, whether to display a filter for it and what type, minimum and maximum values for numeric fields, minimum step for arrow changes, and more. See Class/Client Object Profile and Their Fields.

Client Objects

These are copies of a class but with their own profile for the class itself and its fields, while the set of fields remains the same. This is primarily necessary for the frontend, for example, to prohibit creation in a table while allowing it in a form, or to display different sets of fields in different tables, or to create one table for one role and another for a different one.

On the frontend, client objects are usually associated with a table, form, or frame, which, relying on the profile and layout (relevant for forms and frames), create components on the page. At the same time, they can have their own custom JS code, allowing for different ways of interacting with data and calling various methods.

Application Logic Location

Most of the application logic should be located in the methods of the aforementioned classes. The entry point for any process is some method of a specific class (via the internal API, see below).

There may be some external mechanisms, such as integrations with payment systems, where the implementation of integration with a specific system is moved to a separate location: /modules/payment_system/instances/<payment_system_name>. However, a specific instance overrides or supplements the method of the universal payment system class in /modules/payment_system/instances/index.ts, whose methods are called from the methods of the Ps_transaction entity (the class for transactions with external payment systems). Using the example of payment system integrations, there is a second entry point when the payment system accesses the system URL to notify about the success or failure of a payment. Here we have /routes/index.ts, where the response is processed and also redirected to Ps_transaction.result, which triggers completion in the required payment system instance via its result method. This method, in turn (after parsing and signature verification), calls the final super.result method of the parent payment system integration class (/modules/payment_system/instances/index.ts), which saves the result using the same internal API.

This example shows that various external modules only handle work specific to them, and then control is passed back to the main entity methods.

Calling Methods via Internal API

All methods, both of other classes and the same class, should be called via the internal API.

Sometimes, within a method, if you need to call a method of the same class, you can call it via this. However, you must be aware that in this case, the called method will not be placed in the call chain and will not be logged separately. Locking and unlocking will still work, but unlocking will only occur when the original method that passed through the API completes (if locks are present). It is better to avoid such calls unless you specifically want to avoid excessive logging and fully understand what you are doing.

To access methods of other classes, you should always use the internal API, as simply creating a new instance and calling its method is not sufficient for proper operation.

For more details on what happens in the internal API, see “Internal API in Detail”.

Using Dictionaries instead of Enums

We do not use enums; we use dictionaries of the ds_ type (separately see naming conventions, syntax, etc.).

Queries can be built using sysname; the Core will independently retrieve the ID from the cache or the database and substitute it into the original query.

Creating or modifying a record can also be done using sysname instead of ID; the Core will also handle all the work for you.

In summary, you can freely work with sysname without worrying about substituting the corresponding numeric ID in the database.

All sysname values in dictionaries are written in UPPERCASE, as they stand out clearly in the code. In the future, it might be worth improving the Core so that dictionary content is automatically synchronized into TS types in the corresponding class structure file; then you could use something like ds_order_status.CREATED instead of a string. This would increase code control.

Standardized Input and Output Parameters for Each Method

The input parameter for any method of any entity is a request object r corresponding to the IAPIRequest interface, which contains information about the chain, the client, the specific connection, system request parameters, the requested class and method, and method parameters.

Any method must return an object corresponding to the IAPIResponse interface (sometimes you may see that the output parameter must correspond to IError; this is an outdated form, but IAPIResponse extends IError, and in fact, all responses correspond to both).

To comply with this interface, one of three error/success classes is used: UserOk (used for successful responses), UserError (for user-facing errors, such as “Item out of stock” or “Incorrect password”), and MyError (used for internal errors, such as incorrectly passed parameters—not user-defined, but those passed by code, e.g., “ID not passed” (a user never enters an ID themselves, so this error refers to incorrect method usage)).

All these classes have a code field, which is 0 only for UserOk and a non-zero value for others. For custom methods, by default, the code for MyError and UserError will be the same (-1000). Specific codes are used only for certain conditions primarily used for Core mechanisms. For example, if a request is unauthorized or a session has expired, the code will be -4, and for lack of access, the code is 11. Again, only some Core mechanisms use these codes, and they are not used when developing logic (except for checking for 0). You can see existing codes in /error/error.js (many may be outdated).

A response might look like this:

return new UserOk('Everything is fine, payload is in data', {calculatedAmount: 10, hint:'Calculated for 5 rows'})

Then this data can be retrieved in res.data. For example, console.log(res.data.calculatedAmount) will output 10.

An error response can also contain a payload (data field), which is useful for including the error that occurred and the parameters with which the calling method was invoked.

There are two ways to handle such an error:

Either pass it as is, possibly supplementing and/or changing the text, but preserving the instance and, accordingly, the error type:

if (res.code) return res

There are methods for supplementing an error object: setMsg and setData. setMsg accepts new text and, optionally, an object as a second parameter to supplement data. The second method, setData, only accepts an object to supplement data.

// get row
p = {id: params.id}
res = await r.api(Deal, 'getById', prepareP(p))
if (res.code) {
   // return res
   // return res.setMsg('New error text', {p})
   return res.setData({p})
}

Or create a new one with new text and possibly a new type, and include the error and parameters in data.

p = {id: params.id}
res = await r.api(Deal, 'getById', prepareP(p))
if (res.code) {
   return new UserError('Maybe someone deleted the document, refresh the table and try again.',
       {res, p})
}

No Throws

The Core concept is that methods never throw an error; instead, they return a UserError or MyError object. This principle should be followed when writing your own methods.

If a method uses any built-in or third-party function that might throw an error, it should be wrapped in try/catch, and the catch block should return an IAPIResponse instance.

Naturally, at the top level of the internal API, as well as higher up at the Express level, there is a trap for unforeseen errors, but this is an emergency mechanism.

Hence the conclusion: always check res.code.

It may seem that this approach complicates code writing due to the need to perform this check after every method call via the internal API. However, in return, we gain the following advantages:

A classic Exception provides a stack trace, but considering all optimizations/minifications, calls via anonymous functions, and a large number of intermediate functions in general, reading this stack is practically impossible, and its logging usually cuts off most of the data. The mechanism used in the Core provides a clear chain of method calls and responses at each level, with additional data if specified. That is, we know that method A was called with certain parameters and returned a certain response; inside, it called method B with other parameters, and its response is also available, and so on.

The second advantage (according to the Core’s creators) is that error handling is required at every call level (as mentioned in the previous point), but with Exception throws, this turns into bulky try/catch blocks instead of a short:

if (res.code) return res.setData({p})

or more comprehensive:

if (res.code) return res.setMsg('New text', {p, anotherAdditionalInfo:{abc:123}})

Principle of Writing Any Method

A documenting comment should be placed before the method, sufficient to understand exactly what the method does and what features it has. Describing parameters in it is not required, as the automatically generated JSDoc style will only indicate the r parameter, and the developer is only interested in the contents of r.data.params. They will be described using generics, see the point below.

Input and output parameters—specifically the parts defining a specific method—are documented in generics for IAPIRequest and IAPIResponse, respectively. Comments for parameters can also be included there in single-line or multi-line style. An example can be seen in the code below.

The method code consists of roughly 3 blocks:

Checking input parameters and defining method-level variables needed later.

Method parameters are located in r.data.params. Request parameters (less frequently needed as they are system parameters used by the Core) are located in r.params. For more details, see the IAPIRequest description.

Typically, two variables, p and res, are defined at the beginning and will be reused multiple times for method calls via the internal API.

Main logic consisting of sequential calls to required methods, checks, and calculations.

Usually, this involves retrieving data, performing checks, calling other methods, comparisons, data synchronization, generation, and saving changes to the database.

Many developers like to create a separate method for every micro-task, but we adhere to the concept that logic should be moved primarily if it is used by multiple methods or if a method or file grows significantly.

Returning success.

This block is somewhat nominal, as a method can terminate earlier under certain conditions with either success or failure. But in any case, there must be a final return at the end.

Method example:

/**
* Method will update the description of the deal by setting the current time,
* after which it will prepare the pdf
*/
async doDoc(r: IAPIRequest<{
   id: number
}>): Promise<IAPIResponse<{
   timeOfOperation?:string // Time that will be added to the description
   /*
   * Url to the generated pdf
    */
   pdfLink:string
}>> {
   const rParams: IAPIParams = r.params
   const params: IAPIQueryParams = r.data.params

   const {id} = params
   if (isNaN(+id)) return new MyError('id not passed')

   let p
   let res: IAPIResponse

   let pdfLink = ''
   const timeOfOperation = toUserFriendlyDateTime()

   // get row
   p = {
       id,
       columns:['id', 'status_sysname'],
   }
   res = await r.api(Deal, 'getById', prepareP(p))
   if (res.code) {
       return new UserError('Maybe someone deleted the deal, update the table and try again.', {res, p})
   }
   const row = res.data.rows[0]

   if (row.status_sysname !== 'READY') {
       return new UserError('The deal is not ready yet', {row})
   }

   // Update the deal description
   p = {
       id:row.id,
       description: `Test_description_${timeOfOperation}`,
   }
   res = await r.api(Deal, 'modify', prepareP(p))
   if (res.code) res.setMsg('Failed to update the deal description', {p})

   // Creating pdf...

   return new UserOk('Pdf created', {timeOfOperation, pdfLink})
}

Splitting methods into separate files. Key project entities usually have a large number of methods, and they should be moved to separate files, categorized by logic. Entity files are located in /classes/. For example, /classes/Deal.ts. Some methods can be moved to separate files and placed in a directory with the same name (which needs to be created), e.g., /classes/Deal/someMethods.ts.

Since methods moved to a separate file must not lose access to the context (this), it should be passed. One way:

A method in a separate file might look like this:

export async function prepareReceipt(this: Payment, r: IAPIRequest<{
   ids: number[] // Payment IDs. Multiple can be specified to combine into one receipt
}>): Promise<IAPIResponse<{
   receiptItems:{
       name:string
       price: number
       quantity: number
       amount: number
   }[]
}>> {
…

And in the class itself, we only define the method and call the external one using call:

/**
* Prepares payment items for a receipt. The method considers that specific sub-items 
* for the receipt may be specified for an order position. The method will adjust 
* the cost of such sub-items to the price in the payment.
*/
async prepareReceipt(r: IAPIRequest<{
   ids: number[] // Payment IDs. Multiple can be specified to combine into one receipt
}>): Promise<IAPIResponse<{
   receiptItems:{
       name:string
       price: number
       quantity: number
       amount: number
   }[]
}>> {
   return await prepareReceipt.call(this, r)
}

CRUD. In the Core, it’s AddGetModifyRemove

Every class has these four methods (and others) for working with the database.

Described in more detail in “Base Class Methods”.

Do Not Directly Access the DB. Do Not Write Manual Queries

In most cases, a developer might need to access the database via third-party programs only to create or upload a dump. Occasionally, it may be necessary to inspect data during debugging. Very rarely, manipulations of structure or data might be required if major errors were made during development and a portion needs to be reworked, and Core mechanisms fail due to hung data or something else.

Naming Conventions, Syntax, etc.

Dictionary Naming: d_ and ds_

The Core provides two types of dictionaries.

System dictionaries “ds_” (dictionary system). Used for dictionaries that can change only during development, not during operation. For example, ds_order_status or ds_payment_type.

Dictionaries “d_” (dictionary). Can change during system operation. For example, d_product_category.

Sometimes the boundary can be blurred; for example, language can be either d_ or ds_, as the dictionary may be fully populated immediately or supplemented as needed.

This naming helps with data migration, where ds_ dictionaries are always transferred, while d_ may be transferred once or not at all. Additionally, this allows for better visual orientation in tables during deep debugging or data merging. Such entities also automatically appear in the “Dictionaries” or “System Dictionaries” menus.

Field Types. Used in tables.json during description

id - bigint(20)

Often bigint is used for IDs even for small dictionaries, like ds_gender. We didn’t pay much attention to this previously, but it’s better to use the most suitable type. See the Structure Field section on how to fill tables.json.

You can specify other dimensions, but in that case, remember that when you use a field as a foreign key to another entity, it must be of the same type.

![][image2]

Checkbox - tinyint(1)

Always use this type and length for boolean fields. This is how it’s provided in MariaDB, and this indicator is what the Core relies on for various actions.

Other types correspond to MariaDB types.

Back-office and Frontend. How to Organize

Implementation Options

In many projects, the main Core frontend serves as the back-office, administrative panel, and end-user interface. The Core frontend includes all interfaces for development (managing profiles, menus, permissions, system dictionaries…), as well as for the system administrator (managing users and their access, project-related settings), and for end users (interfaces for the end user are built within a specific project).

In this case, it is primarily a Single Page Application (SPA).

This implementation is available out of the box.

The second option is to write the project frontend separately, whether it’s a website, ERP system, or something else. Then the back-office remains for the developer and system administrators (optionally, as a separate frontend can be written for them too), and the end-user frontend is written separately (on any framework or natively) and communicates with the backend via API (read about connection types below).

Another option is to write a Multi-Page Application (MPA) with Server-Side Rendering (SSR) using the routing provided by the Express framework (part of the Core) and a templating engine like Jade (Pug).

You can also combine the first and third approaches and write the frontend on a separate Core instance using SSR. This allows the “website” instance to handle only the website while maintaining a connection to the backend (a separate instance where the main project logic is implemented).

For multi-page solutions, as well as other external requests, standard Express framework routing is used, from which the internal API can be invoked. See routing for more details.

Connecting to the Backend

The built-in frontend uses our connection library available on npm: go_core_query (https://www.npmjs.com/package/go_core_query), which provides a persistent socket connection to the server, can handle unauthorized request responses (session expired) and redirect to the login page (this action can be overridden). It provides two methods for communicating with the server: via an asynchronous api function (highly similar to the server-side internal API) and, of course, communication via emit/on—everything provided by socket.io. Most logic is usually implemented via api, as the application architecture is much more transparent with a request-response pattern, but for some tasks, the socket can be used as is (to provide two-way communication).

The api function also works via sockets, not fetch, but the developer doesn’t need to worry about that.

To see requests and responses in the console, debug mode must be enabled: call debug(); in the (browser) console; you should see a message about the debug mode change; then refresh the page.

In your applications, you can also use this library on both the frontend and backend, configuring it for your needs. See the library documentation for more details.

Besides the library, you can use a simple HTTP(S) API via fetch or other mechanisms. You will need to perform an authorization request, in response to which you will receive a JWT that must be added to the Authorization header: Authorization: Bearer <Token> for subsequent requests. If the session expires or is manually invalidated, you will receive a standard response object (IAPIResponse) with code “-4”, indicating that you should log in again.

Mobile Application

A mobile application can be written in anything, such as ReactNative. A connection library is available for RN applications just like for the regular frontend. In other cases, the HTTP(S) API will suit you. More details…