Open in app

Sign in

Write

Sign in

File Connector in Mule 4 -Deep Dive Part -I

Priyanka Paul
8 min readApr 21, 2021

--

We generally use File Connector in mule applications for managing files and folders on a locally mounted file system.

File connector support for common FTP operations such as creating directories and copying, moving, renaming, and deleting files. Based on the different FTP operations, we can route to each connector (same set of operations available )like copy, creating new directory , delete, read and write.
It’s also support for locking files and file matching functionality(Matching rules).

It mainly has the below key features :

  • List : List all the files from a given directory
  • Copy: Copies a file from one directory to another
  • Create directory : creates new directory
  • Delete: deletes a file
  • Move: Moves a file from one directory to another
  • On new or Updated files: Triggers when a new file created in a directory
  • Read: Obtain the content and meta data of a file at a given path
  • Write: Writes the content in a file pointed to given directory or path
  • Rename: Renames a file

In this article will try to take you on a deep dive of each of the File connector (Operation based) with real time examples:

LIST:

Use-case 1: List all the files and folder information from a given directory

If you want to retrieve both files and folder information present at the given path, use List connector with file matcher rules as “Directories ” and “Regular files” values as “INCLUDES” which is default configuration .

This use case is handy when you need both the folder and file information for a given path.

Sample code:

<sub-flow name="both-file-and-directories-information-Sub_Flow" doc:id="6f8cd4c5-64f5-407a-adee-6266a989996e" >
        <file:list doc:name="list-both-file-and-folder-info" doc:id="80921088-81e3-4f66-b6d8-0994d7117fa9" directoryPath="/Users/ppaul/Documents/integration-inputs">
            <file:matcher/>
        </file:list>
        <ee:transform doc:name="both file-directories-payload" doc:id="30a06be4-cd18-4bcf-ba75-1bd313f8f48c">
            <ee:message>
                <ee:set-payload><![CDATA[%dw 2.0
output application/json
---

    payload map((item,index) ->
        {
            fileName:item.attributes.fileName,
            fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            filePath:item.attributes.path,
            fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            
        })
    ]]></ee:set-payload>
            </ee:message>
        </ee:transform>
        <logger level="INFO" doc:name="Logger" doc:id="a695b412-a743-430b-8c7f-68e513b73c42" message="Both file and directories listing done!!"/>
    </sub-flow>
Sample Outputs : 
 [
 {
 "fileName": "st.json", (file)
 "fileCreateTimeStamp": "2021-04-10-002547",
 "filePath": "/Users/ppaul/Documents/integration-inputs/st.json",
 "fileLastModified": "2021-04-10-003056"
 },
 {
 "fileName": "f2", (folder)
 "fileCreateTimeStamp": "2021-04-10-211610",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2",
 "fileLastModified": "2021-04-10-211717"
 },
 {
 "fileName": "file1.json",
 "fileCreateTimeStamp": "2021-04-08-155817",
 "filePath": "/Users/ppaul/Documents/integration-inputs/file1.json",
 "fileLastModified": "2021-04-08-155819"
 }
 ]

Use-case 2: List all the files information (without folder ) from a given directory

If you specifically want to retrieve only list of files(excluding folder information ) present at any given path, use List connector with file matcher rules as “Directories ” values as “EXCLUDES”and “Regular files” values as “REQUIRE”during connector configuration.

This use case is convenient when you want to check what are the files (excluding directories /folders) available for a given path.

Sample code:

<sub-flow name="file-list-without-directories-Sub_Flow" doc:id="5a4261b0-8b80-4ef3-95c9-f68ffdde6a9e" >
        <file:list doc:name="file-list-without-directories" doc:id="4f1df50e-e3cd-42f7-8dbc-baaba66818ac" directoryPath="/Users/ppaul/Documents/integration-inputs" recursive="true">
            <file:matcher directories="EXCLUDE" regularFiles="REQUIRE"/>
        </file:list>
        <ee:transform doc:name="file-list-payload" doc:id="e50e0e7a-59e9-45fa-8a60-1778f6507684">
            <ee:message>
                <ee:set-payload><![CDATA[%dw 2.0
output application/json
---

    payload map((item,index) ->
        {
            fileName:item.attributes.fileName,
            fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            filePath:item.attributes.path,
            fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            
        })
    ]]></ee:set-payload>
            </ee:message>
        </ee:transform>
        <logger level="INFO" doc:name="Logger" doc:id="47b56356-0082-4347-8d91-81de6ff8426b" message="file listing done!!ß"/>
    </sub-flow>Sample Outputs:[
 {
 "fileName": "st.json", (file)
 "fileCreateTimeStamp": "2021-04-10-002547",
 "filePath": "/Users/ppaul/Documents/integration-inputs/st.json",
 "fileLastModified": "2021-04-10-003056"
 },
 {
 "fileName": "p1.json", (file)
 "fileCreateTimeStamp": "2021-04-08-155817",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2/p1.json",
 "fileLastModified": "2021-04-08-155819"
 }
 ]

Use-case 3: List all the directories/folder information( without files) from a given path

If you specifically want to retrieve only list of folder(excluding files information ) present at the given path, use List connector with file matcher rules as “Directories ” values as “REQUIRE” and “Regular files” values as “EXCLUDES” during connector configuration.

This use case is a useful example for someone, who is looking for only directories /folders (excluding files) information available at a given path.

Sample code:

<sub-flow name="directories-information-list-without-file-infoSub_Flow" doc:id="6b9bc351-535e-4d56-9f83-fd25bdf5ee9b" >
        <file:list doc:name="directories-information-list-without-file-info" doc:id="7080e528-033d-4db5-99d3-45833d24221c" directoryPath="/Users/ppaul/Documents/integration-inputs" recursive="true">
            <file:matcher directories="REQUIRE" />
        </file:list>
        <ee:transform doc:name="directories-list-payload" doc:id="c46eb034-c578-45ed-81f2-f0f83bb9be2a">
            <ee:message>
                <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
payload map((item,index) ->
        {
            fileName:item.attributes.fileName,
            fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            filePath:item.attributes.path,
            fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            
        })
    ]]></ee:set-payload>
            </ee:message>
        </ee:transform>
        <logger level="INFO" doc:name="Logger" doc:id="94d3c497-5c0f-4f91-9193-965fd125a138" message="directories listing done !! #[payload]"/>
    </sub-flow>
Sample Outputs:
 
 [
 {
 "fileName": "f3", (folder)
 "fileCreateTimeStamp": "2021-04-10-211712",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2/f3",
 "fileLastModified": "2021-04-10-211722"
 },
 {
 "fileName": "f2", (folder)
 "fileCreateTimeStamp": "2021-04-10-211610",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2",
 "fileLastModified": "2021-04-10-211717"
 },
 {
 "fileName": "f1", (folder)
 "fileCreateTimeStamp": "2021-04-10-211602",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f1",
 "fileLastModified": "2021-04-10-211613"
 }
 ]

Use-case 4: List all the file or directories information irrespective of any subfolder from a given path

If you specifically want to retrieve all the files /directories/ only files/ only directories information irrespective of any subfolders present at any given location, use List connector with the below connector configuration.

If you put “Recursive” value as “True” — It will give you all the list of files present in current folder and all subfolders available on the given path.
Incase you put “Recursive” value as “False ” — This is a default configuration . It will collect all the list of files present in the current directory . It doesn’t look for files present in any of the subfolders available on the given path.


This use case is a handy for someone, who is looking for list of files in different layers / specific layers of directories/folders available at a given path.

Sample Code:

<sub-flow name="file-list-irrespective-of-subfolder-SubFlow" doc:id="32f86b93-a4ee-49c6-bde6-e5f78da18f92" >
        <file:list doc:name="file-list-without-directories" doc:id="efbb6ffc-0425-4557-ad48-989be2dee656" directoryPath="/Users/ppaul/Documents/integration-inputs" recursive="true">
        </file:list>
        <ee:transform doc:name="file-list-payload" doc:id="0aeea01f-ba86-4397-b049-c1e7b3be3946">
            <ee:message>
                <ee:set-payload><![CDATA[%dw 2.0
output application/json
---    payload map((item,index) ->
        {
            fileName:item.attributes.fileName,
            fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            filePath:item.attributes.path,
            fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            
        })
    ]]></ee:set-payload>
            </ee:message>
        </ee:transform>
        <logger level="INFO" doc:name="Logger" doc:id="97ac4c43-030c-4093-b792-335d00080d67" message="file listing done!! #[payload]"/>
    </sub-flow>Sample Output:
 
 [
 {
 "fileName": "q1.json",
 "fileCreateTimeStamp": "2021-04-10-002547",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2/f3/q1.json",
 "fileLastModified": "2021-04-10-003056"
 },
 {
 "fileName": "f3",
 "fileCreateTimeStamp": "2021-04-10-211712",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2/f3",
 "fileLastModified": "2021-04-10-211722"
 },
 {
 "fileName": "f2",
 "fileCreateTimeStamp": "2021-04-10-211610",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f2",
 "fileLastModified": "2021-04-10-211717"
 },
 
 {
 "fileName": "file.json",
 "fileCreateTimeStamp": "2021-04-08-154227",
 "filePath": "/Users/ppaul/Documents/integration-inputs/f1/file.json",
 "fileLastModified": "2021-04-08-155559"
 }
 ]

Use-case 5: List all the files with matching rules from a given path

If you want to retrieve list of the files/directories with matching rules for a given location, use List connector with the below connector configuration.

You can configure the “File Matching Rules” as ”Global reference “ or ”Edit Inline”. If you select “Global Reference“ configure the Matcher with the below matching values .

Example

Filename Pattern=”*.json”
Path pattern=”a?*.{xml,pdf}”
created Since=”2021–04–03T13:21:58+00:00" or “lastRunTimeStamp” as DateTime or “LocalDateTime”
created Until=”2021–04–03T13:21:58+00:00"
updated Since=”2021–04–03T13:21:58+00:00"
updated Until=”2021–04–03T13:21:58+00:00"


It will collect the list of files present in the current directory created Until the given DateTime or Filename Pattern matched as “json/pdf”.

In the below example, I have used an object store to save the lastRunTime and retrieve the same timestamp for retrieving the list of files created since the lastRunTime.

This use case is a handy for someone, who is looking for list of files matching with FileName Patterns or other Matching rules available at a given path.

Sample code:

<sub-flow name="file-list-with-mattching-rules-Sub_Flow" doc:id="eb130aa8-d541-4779-985d-8756bdbb8796" >
       <os:retrieve doc:name="Retrieve key" doc:id="093126c9-57cf-4095-ae38-771bcc7d2c16" key="timeStamp" objectStore="Object_store">
            <os:default-value ><![CDATA[#[now() as  LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss+00:SS"}]]]></os:default-value>
        </os:retrieve>
        <ee:transform doc:name="timeStamp" doc:id="e1165589-452b-4ba9-b526-94f7bb3442fa" >
            <ee:message >
            </ee:message>
            <ee:variables >
                <ee:set-variable variableName="lastRunVal" ><![CDATA[%dw 2.0
output application/java
---
payload as LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss+00:SS"}]]></ee:set-variable>
            </ee:variables>
        </ee:transform>
        <file:list doc:name="List with matching rules" doc:id="4788d2e6-83ce-40ec-87b0-19401090b155" directoryPath="/Users/ppaul/Downloads/pdf_documents_" recursive="true" matcher="Matcher"/>
        <ee:transform doc:name="Transform Message" doc:id="e57b3670-8c17-4005-a485-392ece97e569">
            <ee:message>
                <ee:set-payload><![CDATA[%dw 2.0
output application/json
---

    payload map((item,index) ->
        {
            fileName:item.attributes.fileName,
            fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            filePath:item.attributes.path,
            fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
            
             latruntime_Received: vars.lastRunVal
            
            
        })
    ]]></ee:set-payload>
            </ee:message>
        </ee:transform>
        <ee:transform doc:name="lastruntime" doc:id="125e03df-02a6-4123-8b80-7a7aed238b4b" >
            <ee:message >
            </ee:message>
            <ee:variables >
                <ee:set-variable variableName="lastRunTime" ><![CDATA[%dw 2.0
output application/json
---
now() as  LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss+00:SS"}
        
        ]]></ee:set-variable>
            </ee:variables>
        </ee:transform>
        <os:store doc:name="Store key" doc:id="d7c8ce80-cbe1-48ed-8199-f68fe02c2c8f" key="timeStamp" objectStore="Object_store">
            <os:value ><![CDATA[#[vars.lastRunTime]]]></os:value>
        </os:store>
        <logger level="INFO" doc:name="Logger" doc:id="e046b594-7b01-4caf-a19c-dbf2befc2c7e" message="#[payload]" />
        </sub-flow>Global Matcher :
 
 <file:matcher name="Matcher" doc:name="Matcher" doc:id="fcd4ecfc-6df3-4fa6-a0c3-cb9413b7bf17" 
  createdSince="#[vars.lastRunVal]" directories="EXCLUDE" regularFiles="REQUIRE" filenamePattern="*.pdf"/>

Sample Output:

[
 {
 "fileName": "newlycreatedfile2.pdf",
 "fileCreateTimeStamp": "2021-04-10-002547",
 "filePath": "/Users/ppaul/Downloads/pdf_documents_/newlycreatedfile2.pdf",
 "fileLastModified": "2021-04-14-225247"
 },
 {
 "fileName": "newlycreatedfile3.pdf",
 "fileCreateTimeStamp": "2021-04-10-002547",
 "filePath": "/Users/ppaul/Downloads/pdf_documents_/newlycreatedfile3.pdf",
 "fileLastModified": "2021-04-14-225257"
 }
 ]

Thank you for reading!! Rest of the file connectors will be discussed in the next part ( Part II)

___________________________________________________________________

Mule
Mulesoft
Files
File Sharing
Mulesoft Esb

--

--

Priyanka Paul
Priyanka Paul

Written by Priyanka Paul

156 Followers
6 Following

Hello Techies, I love to learn and grow. Currently working as mulesoft Developer @Salesforce . I have experience in both backend(java) and middleware (MuleSoft)

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams