Welcome to the comprehensive guide on managing bulk data in RebelMouse through importing.
This document elaborates on the procedure to import diverse objects like Users (referred to as Authors), Sections, Images, and Posts, ensuring your site remains vibrant and updated.
Before going through this doc make sure you've read our previous documentation on: Setup your import files. It will help you make sense of the importing and make sure you have your data streamlined for success!
You can import objects, such as Users (also called Authors), Sections, Images and Posts.
The following are the types of imports you can complete, and the objects supported:
- You can create and updateUsers and Posts
- You can create, but not update,Sections and Images
- Getting Started
- Upload File or Choose Feed
- Mapping Step
- One to One Mapping
- Advanced Mapping
- Reusing the most used functions
- Reusing the same mapping
- Ready with the mapping? Move to Validate
- Let's import
Once you've exported the data from your previous CMS and that you set up your files in the expected format, to import into RebelMouse:
- Go to your Importer Dashboard under yourdomain.com/r/tasks/importer in your web browser.
You will be able to see the following screen:
- You can choose any of the objects to start the import process. As we mentioned in the previous article we suggest you to always start with Images.
- Click in the Object you want to start, in this example we will start with Images:
Upload File or Choose Feed
You have two options to import data from:
- Uploading a file
- Using Feed
Check Requirements and Technical Limits for technical limitations
In this step you can choose to upload one of your files to get started (by clicking on the blue area), and if they are well-formatted (not broken) once the file has been uploaded you will see the following screen:
This means you can continue to process to the next step: the mapping phase.
If you receive the following error, please check our Troubleshoot article (coming soon).
Note: We currently only accept JSON feeds
If you select the option Feed URL you will need to add details of a feed you want to pull from:
Url Path for the feed on first input area
Pagination Param (required)
Pagination type is offset or page
Basic Auth User (if any)
Password (if any)
Offset pagination involves specifying a numerical offset to start retrieving data from a certain position in a dataset, allowing you to fetch a specific number of items after that offset.
For example, if you have a list of 100 items and you set an offset of 20, you would retrieve items 21 to 40
Page pagination divides the dataset into fixed-size pages, and users request specific pages to navigate through the data in a structured manner.
For example, if you have a list of 100 items and the pagination is on pages of 20 items each, you would find items 21 to 40 by adding page=2.
Once the file was successfully uploaded or the feed validated then you will move to the following step, mapping:
Goal: The goal of this step is to map each property (field) coming from the original input source (the feed or the file you've uploaded) to the RM accepted format for each Object.
This can be achieved in two ways:
- One to One mapping - where one field is mapped directly to another field. This is assuming the property you are mapping has the same property type as the RM destination property, and it is ready to be imported as it is.
For this goal to be achieved we have 3 main areas on this screen:
Input Properties (left area)
This area on the left will contain a random sample of the max 20 records that were pulled from the files you've uploaded or the feed you've chosen from.
Object Properties (mid area) - Actual Map
This area on the mid area is where you will have to define if you want to do a OneToOne mapping or Code Mapping, and which input field (or fields) will be mapped to which RebelMouse property.
Preview Area (right area)
Once you do the mapping on the mid area, you can then click on preview and have a preview of what the destination Object will look like for the sample records.
Let's work with some examples:
One to One Mapping
Example: Simple Image csv
Import Images using OneToOne Mapping:
Imagine a csv file that looks like the following:
We now imported it, and we are at the Map step, and the screen will look like this:
I will only map image_url (as it is required) and Alt:
When I click in Preview in each of the properties I will see how the property will behave on Preview Area. When I click in Preview all Fields I see how the entire object will look like.
Now what happens when the data is not prepared for importing or is in a different type that the destination accepts? Then we will need to use the Code Mapping:
Code Mapping (Advanced)
For accessing the code Mapping you click on Choose Tool, Code:
Using the same example, where the input field is already prepared and in the type specified for that property on the RM Object, we can achieve the exact same result as the OneToOne tool by doing the following:
Now the entire same approach in a short video:
Interesting to note that:
- Each code area already tells you which data type it expects:
In this case image_url needs to be type string
And if you return a different data type, our code area will let you know:
Type: ‘number’ is not assignable to type ‘string’
Now for a harder example:
In this example we have a csv file that contains only one column: image_tags
And it contains all the information we need:
image_url, alt, caption, etc
So when we upload it, it looks like this:
Now we want to use code to extract the image url and the image alt from the html string coming from input property: image_tags
The way to do it is:
If no errors are displayed you can then test the results by previewing them:
Note: You can use as many input fields as you want, including the entire input object. You can also call as many functions as you want.
- You cannot use external libraries, nor DOM libraries
- You are not allowed to do external HTTP requests
Reusing the most used functions
Find in this doc the latest functions and reuse them!
Reusing the same mapping
If you have a large import task, preparing your data might take some time.
Don’t worry, we have a feature that enables you to download or upload your mapping work easily.
This way, you can save your progress and pick up where you left off, even the next day. It also allows for you to work with different colleagues.
And finally if you have a blocker and need RebelMouse's team assistance you just need to send the object file (Users, Images...) you're importing and the latest mapping file to firstname.lastname@example.org.
To download your current mapping, click the down arrow. To upload a saved mapping, simply click the up arrow.
Ready with the mapping? Move to Validate
The validation step ensures that your uploaded data is properly structured and that all the transformation you've specified on the previous step works well. If any data record doesn't conform, such as a field having an incorrect or untreated type that disrupts the mapping logic, an error will appear to prevent problems.
If you encounter an error on Validate step we recommend you to download the error csv and try to understand how to fix it (on code or on the uploaded file) before continuing to import.
Please refer to our topic on Troubleshooting errors on Validate Step for more information
If validation goes smoothly without errors, you're good to go for the import step. During import, we transfer your carefully prepared, mapped, and transformed data to RM through our public API. Should any issues arise during this phase, it may be due to API limitations, and you can find additional information in our documentation at apis.rebelmouse.com.
Please note that handling large files may extend the process duration. You can monitor progress on the import initial page, and once the import is complete, it will be visible in the import history section.
Please refer to our topic on Troubleshooting errors on Import Step for more information (coming soon) or send us an email to email@example.com with:
- Import file
- Last mapping file you've downloaded on the tool