File Connector in Mule 4 -Deep Dive Part -I

We generally use File Connector in mule applications for managing files and folders on a locally mounted file system.

File connector support for common FTP operations such as creating directories and copying, moving, renaming, and deleting files. Based on the different FTP operations, we can route to each connector (same set of operations available )like copy, creating new directory , delete, read and write.
It’s also support for locking files and file matching functionality(Matching rules).

It mainly has the below key features :

  • List : List all the files from a given directory
  • Copy: Copies a file from one directory to another
  • Create directory : creates new directory
  • Delete: deletes a file
  • Move: Moves a file from one directory to another
  • On new or Updated files: Triggers when a new file created in a directory
  • Read: Obtain the content and meta data of a file at a given path
  • Write: Writes the content in a file pointed to given directory or path
  • Rename: Renames a file

In this article will try to take you on a deep dive of each of the File connector (Operation based) with real time examples:

LIST:

Use-case 1: List all the files and folder information from a given directory

Sample code:

<sub-flow name="both-file-and-directories-information-Sub_Flow" doc:id="6f8cd4c5-64f5-407a-adee-6266a989996e" >
<file:list doc:name="list-both-file-and-folder-info" doc:id="80921088-81e3-4f66-b6d8-0994d7117fa9" directoryPath="/Users/ppaul/Documents/integration-inputs">
<file:matcher/>
</file:list>
<ee:transform doc:name="both file-directories-payload" doc:id="30a06be4-cd18-4bcf-ba75-1bd313f8f48c">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json
---

payload map((item,index) ->
{
fileName:item.attributes.fileName,
fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
filePath:item.attributes.path,
fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},

})
]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="a695b412-a743-430b-8c7f-68e513b73c42" message="Both file and directories listing done!!"/>
</sub-flow>
Sample Outputs : 
[
{
"fileName": "st.json", (file)
"fileCreateTimeStamp": "2021-04-10-002547",
"filePath": "/Users/ppaul/Documents/integration-inputs/st.json",
"fileLastModified": "2021-04-10-003056"
},
{
"fileName": "f2", (folder)
"fileCreateTimeStamp": "2021-04-10-211610",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2",
"fileLastModified": "2021-04-10-211717"
},
{
"fileName": "file1.json",
"fileCreateTimeStamp": "2021-04-08-155817",
"filePath": "/Users/ppaul/Documents/integration-inputs/file1.json",
"fileLastModified": "2021-04-08-155819"
}
]

Use-case 2: List all the files information (without folder ) from a given directory

Sample code:

<sub-flow name="file-list-without-directories-Sub_Flow" doc:id="5a4261b0-8b80-4ef3-95c9-f68ffdde6a9e" >
<file:list doc:name="file-list-without-directories" doc:id="4f1df50e-e3cd-42f7-8dbc-baaba66818ac" directoryPath="/Users/ppaul/Documents/integration-inputs" recursive="true">
<file:matcher directories="EXCLUDE" regularFiles="REQUIRE"/>
</file:list>
<ee:transform doc:name="file-list-payload" doc:id="e50e0e7a-59e9-45fa-8a60-1778f6507684">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json
---

payload map((item,index) ->
{
fileName:item.attributes.fileName,
fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
filePath:item.attributes.path,
fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},

})
]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="47b56356-0082-4347-8d91-81de6ff8426b" message="file listing done!!ß"/>
</sub-flow>
Sample Outputs:[
{
"fileName": "st.json", (file)
"fileCreateTimeStamp": "2021-04-10-002547",
"filePath": "/Users/ppaul/Documents/integration-inputs/st.json",
"fileLastModified": "2021-04-10-003056"
},
{
"fileName": "p1.json", (file)
"fileCreateTimeStamp": "2021-04-08-155817",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2/p1.json",
"fileLastModified": "2021-04-08-155819"
}
]

Use-case 3: List all the directories/folder information( without files) from a given path

Sample code:

<sub-flow name="directories-information-list-without-file-infoSub_Flow" doc:id="6b9bc351-535e-4d56-9f83-fd25bdf5ee9b" >
<file:list doc:name="directories-information-list-without-file-info" doc:id="7080e528-033d-4db5-99d3-45833d24221c" directoryPath="/Users/ppaul/Documents/integration-inputs" recursive="true">
<file:matcher directories="REQUIRE" />
</file:list>
<ee:transform doc:name="directories-list-payload" doc:id="c46eb034-c578-45ed-81f2-f0f83bb9be2a">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json
---
payload map((item,index) ->
{
fileName:item.attributes.fileName,
fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
filePath:item.attributes.path,
fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},

})
]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="94d3c497-5c0f-4f91-9193-965fd125a138" message="directories listing done !! #[payload]"/>
</sub-flow>
Sample Outputs:

[
{
"fileName": "f3", (folder)
"fileCreateTimeStamp": "2021-04-10-211712",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2/f3",
"fileLastModified": "2021-04-10-211722"
},
{
"fileName": "f2", (folder)
"fileCreateTimeStamp": "2021-04-10-211610",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2",
"fileLastModified": "2021-04-10-211717"
},
{
"fileName": "f1", (folder)
"fileCreateTimeStamp": "2021-04-10-211602",
"filePath": "/Users/ppaul/Documents/integration-inputs/f1",
"fileLastModified": "2021-04-10-211613"
}
]

Use-case 4: List all the file or directories information irrespective of any subfolder from a given path

Sample Code:

<sub-flow name="file-list-irrespective-of-subfolder-SubFlow" doc:id="32f86b93-a4ee-49c6-bde6-e5f78da18f92" >
<file:list doc:name="file-list-without-directories" doc:id="efbb6ffc-0425-4557-ad48-989be2dee656" directoryPath="/Users/ppaul/Documents/integration-inputs" recursive="true">
</file:list>
<ee:transform doc:name="file-list-payload" doc:id="0aeea01f-ba86-4397-b049-c1e7b3be3946">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json
---
payload map((item,index) ->
{
fileName:item.attributes.fileName,
fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
filePath:item.attributes.path,
fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},

})
]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="97ac4c43-030c-4093-b792-335d00080d67" message="file listing done!! #[payload]"/>
</sub-flow>
Sample Output:

[
{
"fileName": "q1.json",
"fileCreateTimeStamp": "2021-04-10-002547",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2/f3/q1.json",
"fileLastModified": "2021-04-10-003056"
},
{
"fileName": "f3",
"fileCreateTimeStamp": "2021-04-10-211712",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2/f3",
"fileLastModified": "2021-04-10-211722"
},
{
"fileName": "f2",
"fileCreateTimeStamp": "2021-04-10-211610",
"filePath": "/Users/ppaul/Documents/integration-inputs/f2",
"fileLastModified": "2021-04-10-211717"
},

{
"fileName": "file.json",
"fileCreateTimeStamp": "2021-04-08-154227",
"filePath": "/Users/ppaul/Documents/integration-inputs/f1/file.json",
"fileLastModified": "2021-04-08-155559"
}
]

Use-case 5: List all the files with matching rules from a given path

Sample code:

<sub-flow name="file-list-with-mattching-rules-Sub_Flow" doc:id="eb130aa8-d541-4779-985d-8756bdbb8796" >
<os:retrieve doc:name="Retrieve key" doc:id="093126c9-57cf-4095-ae38-771bcc7d2c16" key="timeStamp" objectStore="Object_store">
<os:default-value ><![CDATA[#[now() as LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss+00:SS"}]]]></os:default-value>
</os:retrieve>
<ee:transform doc:name="timeStamp" doc:id="e1165589-452b-4ba9-b526-94f7bb3442fa" >
<ee:message >
</ee:message>
<ee:variables >
<ee:set-variable variableName="lastRunVal" ><![CDATA[%dw 2.0
output application/java
---
payload as LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss+00:SS"}]]></ee:set-variable>
</ee:variables>
</ee:transform>
<file:list doc:name="List with matching rules" doc:id="4788d2e6-83ce-40ec-87b0-19401090b155" directoryPath="/Users/ppaul/Downloads/pdf_documents_" recursive="true" matcher="Matcher"/>
<ee:transform doc:name="Transform Message" doc:id="e57b3670-8c17-4005-a485-392ece97e569">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json
---

payload map((item,index) ->
{
fileName:item.attributes.fileName,
fileCreateTimeStamp: item.attributes.creationTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},
filePath:item.attributes.path,
fileLastModified: item.attributes.lastModifiedTime as LocalDateTime {format: "YYYY-MM-dd-HHmmss"},

latruntime_Received: vars.lastRunVal


})
]]></ee:set-payload>
</ee:message>
</ee:transform>
<ee:transform doc:name="lastruntime" doc:id="125e03df-02a6-4123-8b80-7a7aed238b4b" >
<ee:message >
</ee:message>
<ee:variables >
<ee:set-variable variableName="lastRunTime" ><![CDATA[%dw 2.0
output application/json
---
now() as LocalDateTime {format: "yyyy-MM-dd'T'HH:mm:ss+00:SS"}

]]></ee:set-variable>
</ee:variables>
</ee:transform>
<os:store doc:name="Store key" doc:id="d7c8ce80-cbe1-48ed-8199-f68fe02c2c8f" key="timeStamp" objectStore="Object_store">
<os:value ><![CDATA[#[vars.lastRunTime]]]></os:value>
</os:store>
<logger level="INFO" doc:name="Logger" doc:id="e046b594-7b01-4caf-a19c-dbf2befc2c7e" message="#[payload]" />
</sub-flow>
Global Matcher :

<file:matcher name="Matcher" doc:name="Matcher" doc:id="fcd4ecfc-6df3-4fa6-a0c3-cb9413b7bf17"
createdSince="#[vars.lastRunVal]" directories="EXCLUDE" regularFiles="REQUIRE" filenamePattern="*.pdf"/>

Sample Output:

[
{
"fileName": "newlycreatedfile2.pdf",
"fileCreateTimeStamp": "2021-04-10-002547",
"filePath": "/Users/ppaul/Downloads/pdf_documents_/newlycreatedfile2.pdf",
"fileLastModified": "2021-04-14-225247"
},
{
"fileName": "newlycreatedfile3.pdf",
"fileCreateTimeStamp": "2021-04-10-002547",
"filePath": "/Users/ppaul/Downloads/pdf_documents_/newlycreatedfile3.pdf",
"fileLastModified": "2021-04-14-225257"
}
]

Thank you for reading!! Rest of the file connectors will be discussed in the next part ( Part II)

___________________________________________________________________

Hello Techies, I love to learn and grow. Currently working as mulesoft Developer @Salesforce . I have experience in both backend(java) and middleware (MuleSoft)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store