data exchange
Patterns as operational constructs.
data exchange, why is it critical?
There are a lot of questions to answer:
📚 Information data is describing?
⚙ Relationships data elements?
🎭 Who is using data for what proces?
⚖ Inventory information being used ?
🔰 Somewhere in a loop of patterns ..
Most logical back reference:
previous.
Contents
Progress
- 2020 week:02
- Rebuilding the exchange page.
Focus on the technical topics in exchange of information.
Duality service requests
sdlc: Needed to integrated business application systems when those assembly lines are separated.
Reliability of the information (data) and delivering that in time is critical.
bianl: Analyses of the business process for answering improvement questions is not critical.
Analysing data for creating new information that is mandatory and must be delivered with a deadline is critical.
Why and how doing information exchange.
The need for data that is collected or being derivate at another process getting into the local process is a normal requirement.
There must be a list of what is needed.
Exchange, The sending on some request.
The process flow of delivering data is clear.
- When should it be send
- What should be send
- How (who is the executor) should be send
- To who, the recipient
With an easy object is that could be like the following figure.
Exchange, The Delivery end-point question.
The process of handling the deliveries/parcels not. Questions:
- Who´s is handling the receiving point
- Is there a check of not putting unwanted stuff
- Will there be a quality check on the delivery/parcel
Use of a secured location secured storage to get mail/parcel
Having a safe place fore receiving the goods is for additional safety.
The secured storage approach.
There are a lot of tools for a secured file transfer.
-
⚙ FTP, SFTP, XFB, XCOM. Cryptshare etc.
A gap in engineering is usually the authorisation of the receiving storage area.
This is an hot issue when using a multi tenancy service.
Multi tenancy is sharing a lot of things reducing resources and software releases complexity.
Requirement: solving multi tenancy while not adding too much complexity.
Using OS controls implementing limited access security.
⚠ Assuming the authorisation could be done by describing them by files is a pitfall.
The files are moved and deleted at a moment in time. When the access is bound to a file it can get deleted also.
💡 using a well defined ACL (access control list) on a directory will result in a reliable reusable container in the same way of the controlled receiving of parcels.
The Unix (DAC) Solution:
- Acknowledge the need for a logical <dtap> segregation
- Define directory “business receiver” authorised by user:group <applid>_<dtap>:<applid>_<dtap>
- Define the owner of this directory: the non personal account impersonating the application (data subjects) of the receiving environment eg <applid>_<dtap>.
- Limit access to the directories to a white list of accounts (using roles - group)
- Have the execute rights on the directory set. It has not the meaning of execution but the "change directory" action.
- ❗ Have the sticky bit on the directory set.
It has no meaning for execution but for who can delete files in the directory.
- ❗ Have the Setgid bit on the directory set.
It has the meaning new created files will get that group associated with the file not the one primary group of the creator (owner).
This works even when the creator is not a member of that group.
- Access rights on the directory: world-wide open (3757 or 3746), setguid and sticky-bit
The idea is that files are registered in directories an by that will need authorisation to those directories for any access. There are three different options of access as the owner group and others have different rights.
This is Standard solution in Unix. Commonly used at “/tmp” or Unix-mail.
It is also common knowledge fore auditors (search internet with the keywords)
In a table representation:
| | owner:
group | appl01_bd:
appl01_bd | appl01_bt:
appl01_bt | appl01_ba:
appl01_ba | appl01_bp:
appl01_bp |
| Environment | - | D | T | A | P |
| /data/…/xchn_d/appl01_d/.. | D g
D o | R
RW */3 | | | |
| /data/…/xchn_t/appl01_t/.. | D g
D o | | R
RW */3 | | |
| /data/…/xchn_a/appl01_a/.. | D g
D o | | | R
RW */3 | |
| /data/…/xchn_p/appl01_p/.. | D g
D o | | | | R
RW */3 |
Remarks:
- * "D g" for access directory by a group, "D o" directory access for others (often assumed as world)
- */3 Public access to the map/directory in combination
⚠ The fourth number in front (3) Setgroupid and sticky bit is indispensable. with setgid sticky-bit.
The advantage of this approach are:
- Eliminates the need fore a dedicated "sharing group" for every possible combination. There is a reduce in complexity avoiding that.
- Just an owner can delete files.
It must be the owner of the directory or the owner of the file
Disadvantages are:
- A lot of non personal high privileged accounts and groups are set up just for a receiving container functionality.
- Just an owner can delete the files.
Limited white list receiving storage.
⚖ What is achieved is that files can get stored by other accounts and the intended receiver can read and also delete files. The deletion is only for the owner of the file (sender) and owner of the directory possible.
💣 More detailed questions of possible abuse should get covered.
Shortcomings of basic receiving storage
- Everyone is able to post files at the destination.
This is bypassing integrity as unauthorised placing of file is possible.
- Everyone is able to see what files are there in the exchange receiving area.
- Storage and mount point responsibility is not clear defined
Abuse of storage by (personal keys) users is possible
- The way of how retention policies should be solved is not clear.
How long should a copy of files be available?
- Monitoring resource usage is not clear.
It could possible to block all tenants needing the data storage exchange.
Defining administration role for receiving storage
There is an administrator role defined for managing the receiving storage exchange area needed.
An impersonated name and group could be: user:group xchn_<dtap>:xchn_<dtap>
Any account that wanted to use the storage exchange area is required to be member of the group. Note the required strict dtap segregation is implied by using separated accounts and groups.
In a table representation:
| | key:
group | xchn_d:
xchn_d | xchn_t:
xchn_t | xchn_a:
xchn_a | xchn_p:
xchn_p |
| Environment | | D | T | A | P |
| /data/../xchn_d | D g
D o | RX
--- | | | |
| /data/../xchn_t | D g
D o | | RX
--- | | |
| /data/../xchn_a | D g
D o | | | RX
--- | |
| /data/../xchn_p | D g
D o | | | | RX
--- |
A living environment with existing files.
The environment is set up with empty directories. When it is used files will exist. The DAC attributes will vary when they are created.
Moving a file will need to copy the file and delete the received file by the reciever as moving would keep the sender owner atttibutes.
In a table representation:
| | D owner:
D group | appl01_bd:
appl01_bd | appl01_bt:
appl01_bt | appl01_ba:
appl01_ba | appl01_bp:
appl01_bp |
| | F owner:
F group | */1 ?:
appl01_bd | */1 ?:
appl01_bt | */1 ?:
appl01_ba | */1 ?:
appl01_bp |
| Environment | - | D | T | A | P |
| /data/…/xchn_d/appl01_d | D g
D o | R
RW */3 | | | |
| /data/…/xchn_d/appl01_d/.. | F g
F o | R */4
- */5 | | | |
| /data/…/xchn_t/appl01_t | D g
D o | | R
RW */3 | | |
| /data/…/xchn_t/appl01_t/.. | F g
F o | | R */4
- */5 | | |
| /data/…/xchn_a/appl01_a | D g
D o | | | R
RW */3 | |
| /data/…/xchn_a/appl01_a/.. | F g
F o | | | R */4
- */5 | |
| /data/…/xchn_p/appl01_p | D g
D o | | | | R
RW */3 |
| /data/…/xchn_p/appl01_p/.. | F g
F o | | | | R */4
- */5 |
Remarks:
- * "D g" for access directory by a group, "D o" access directory for others (often assumed as world)
- * "F g" for access files by a group, "F o" access files for others (often assumed as world)
- */1 which files are place is up to the business.
The prediction of the physical names when how by whom is impossible
- */2. The ownership will be a result of an action from posting a file.
- */3 Limited Others (not Public) access to the map/directory in combination
⚠ The fourth number in front (3) Setgid and sticky bit is indispensable.
- */4. The senders account should files make readable at group level. The receiving account is the owner of the directory Not the owner of files
- */5. All other keys in the system are excluded of access to files
Administrate receiving storage.
⚖ Achieved: intended functionality on exchanging files by sender and receiver.
⚖ Safety: Limited to only impersonated accounts and only those granted for this functionality.
⚠ Not yet mentioned are administrative actions needed to be in place for this.
Managing storage size capacity and availability
What is needed is someone that will monitor the machine capacity usage.
⚠ Too many big files at the same moment could cause capacity problems. Communicating for solutions in capacity is necessary.
Files should get deleted or archived to another location after being processed.
Security incidents should have some follow up (IRC).
Defining new receive locations
When a new directory needs to get defined that could be done using root access rights. Another way is:
- As impersonated owner (xchn_<dtap>) account: open up the directory right to create a new directory.
- As impersonated owner of the receiving application: define the new directory in the agreed setting. Naming and access rights.
- As impersonated owner (xchn_<dtap>) account: close the directory right to create a new directory.
All other needed autorisations are based on group meberships. Groupmemberships are managed by a central dpeartment
Reference when an auditor does information requests
It is a standard approach within Linux(Unix) exists as used on maps like "/tmp" and "cron" "mail"
See:
rights-management source: debian_handbook.info or>
file attributes explanation source: linuxconfig.org
When used on a directory, instead, the setgid bit alters the standard behaviour so that the group of the files created inside said directory, will not be that of the user who created them, but that of the parent directory itself.
This is often used to ease the sharing of files.
Allowing other (public) rights in the directories (RWX) can raise questions. The limitations by the hierarchy and those special bits on a directory not generic understood well.
Chapter 49. Security and SELinux source: RedHat
🤔 A path traversal attack is using the lack of security in the system. Running a process that is not having least privileges set.
This pattern is closing any traversal attack gap by implementing well defined security.
Left attention points
- There are some minor differences with the physical delivery.
The impersonated owner of the file keeps to have full control and can withdraw files at any moment.
Within the limited members of an exchange group the files names of others are visible. Not the content or whatsoever just the names.
- The process of cleaning up old files, archiving files and monitoring must be organized.
It can easily get out of sight resulting in too many files and complexity in solving under pressure.
Defining the chain - Transformations.
Pattern: information exchange & data security
Defining an exchange storage area by:
- Having a controlled administrated entry.
Using a name like xchn_<dtap>
- Set up well defined access controls guided by the xchn administrator for receiving and sending parties.
- Allow each party for their own sub directories.
Although there is no additional software and scripting must be build in house this pattern can seen as an "application".
With (application id 019, * parallel line the name for receiving application name
A name could be:
<dtap>/B019_/&let019N>/L<*>_Ldta/%*
Remarks:
- Auditing and acceptance for this not well known approach can be a challenge.
Conflicting other Patterns.
This approach is a very simple solution at host security controls level.
⚠ It is often felt as threatening projects running commercial products, instead of having a (os level) secured environment.
Dependencies other patterns
This page is a pattern on naming conventions.
Within the scope of metadata there are more patterns like exchanging information (data), building private metadata and securing a complete environment.
🔰 Somewhere in a loop of patterns ..
Most logical back reference:
previous.