Re: Import Transactions

Wiki-ed the info.

Re: Import Transactions

Hi,
I have adapted the extension to import multi line sales orders, and sales invoices from a spreadsheet. Invoices are either cash sale or credit sale.

The extension is working fine but I am getting a Cart: Read warning (yellow) whilst importing the Direct Sales Invoices.  The sales orders import fine. 

I do not want to rewrite the Cart Class simply to avoid a warning.

I am using the attached adapted British coa with a service item 001 and a product item 002. One customer connected to branch one and a second customer connected to branch 2.

Any ideas would be appreciated. See attached files please.

Regards

Ross

Re: Import Transactions

Your attachment is attached to this post.

Post's attachments

ImpTrans.zip 43.2 kb, 9 downloads since 2015-04-28 

You don't have the permssions to download the attachments of this post.

Re: Import Transactions

Hi raddison,

I am using your import transaction module with some limited success. Followed the instructions faithfully and all the bank statement for both customers' & suppliers' data checked ok during the trial run-check before importing.

For the customers/bank receipts, the data was imported correctly. For the suppliers/bank payments, the suppliers' name is missing. To complete the process, I have to adjust the bank payments manually to include the suppliers' names.

Any advice?

peace, peter.

Re: Import Transactions

@peter: please compare your CoA with that of the standard sql/en_US-new.sql for field order differences and correct them to match the latter. Many CoAs are "broken" and some like the en_GB are corrected in my unofficial GitHub Repo.

Re: Import Transactions

@apmuthu, thanks for the advice. Will check structure.

peace, peter.

Re: Import Transactions

Hi,
I have updated the extension to import Sales Orders and Invoices.  The code is available on my github repo.
https://github.com/rossaddison/import_transactions.

Multiline or singleline invoices can be imported. Spreadsheet templates are available under the templates folder for both multiline and single line invoices.

Invoices can be either cash or credit. The payment id lookup tool should be used to determine what number you will associate with either cash or credit invoices.  For cash invoices I have used the payment id of 4, and for credit invoices I have arbitrarily used 1 instead of 2 or 3.

33 (edited by apmuthu 05/10/2015 05:57:54 pm)

Re: Import Transactions

In modules/import_transactions/includes/import_transactions.inc replace line:

if ($person_id != null)

with

if (!empty($person_id))

In modules/import_transactions/import_transactions.php replace line:

$prev_ref = 9999;

with

$prev_ref = -1;

so that a practically impossible value is used for initialisation.

My Unofficial GitHub repo has been updated with white space fixes as well and the extra comma in the last line of templates/payment_en_GB.csv has been eliminated.

You may also further put in the constants for balance of $type == 0, 1, 2 comparisons.

Post's attachments

import_transactions_2.3.24-1.zip 49.8 kb, 8 downloads since 2015-05-10 

You don't have the permssions to download the attachments of this post.

Re: Import Transactions

I have made the adjustments.

Regards
Ross

Re: Import Transactions

In your file templates/payment_en_GB.csv, please delete the last comma. There are 11 fields separated by 10 commas but the last line has an extra comma.

Re: Import Transactions

Done

Re: Import Transactions

@joe: PKG Repo update needed for this extension.

Re: Import Transactions

I must be missing something here smile

You mention bank statement import and have a template available but there is no bank statement import available in "Import Type" dropdown

I see:

<option  value='0'>General Journal Processing</option>
<option  value='2'>Deposit Processing</option>
<option  value='1'>Payment Processing</option>
<option  value='30'>Sales Order Processing</option>
<option  value='10'>Direct Sales Invoice Processing </option>

None of those options work with my csv based on your template

https://raw.githubusercontent.com/rossaddison/import_transactions/master/templates/bank_en_GB.csv

Re: Import Transactions

For Bank Statement Processing:
Check the bank statement checkbox below the drop down box.
Your csv must have the two amount columns. One for deposits. One for payments.
Which is typical of any bank statement.

Remember this is an adapted Bank Statement csv file that you are importing.

So the assumption is:
1. Download your bank statement from your bank in csv format.
2. Incorporate additional columns in this spreadsheet according to the template.
3. Complete your account code designations to the transactions in your bank.csv including tax codes.
4. Import the csv.

So you do the all account code and vat code designations for each line or transaction on the bank statement before you import.

The first few columns of the spreadsheet/csv must be identical to your bank statement.
The last few columns incorporate account and tax allocation.

So this spreadsheet/csv that you have created has your account and vat allocations in addition to the bank statement.

Essentially you are doing your account and vat  code designations before the import. Which I personally find much easier than any other commercial software packages available on the market. Complete all your account code designations to each transaction before you do the import.

Process the same csv twice.
Once with Deposit Processing.
Next with Payment Processing.

Hope that clears a few things up.

Re: Import Transactions

Hi all,

I'm having some difficulties importing journals. The file contains multiple journals. Each journal contains multiple lines.

reference,date,memo,amount,accountcode,taxtype,dim1_ref,dim2_ref,person_type_id (supplier=1 customer=2),person_id
JNL1,03-07-2014,test,23.30,8000,0,,,,
JNL1,03-07-2014,test,173.90,8001,0,,,,
JNL1,03-07-2014,test,-16.00,1700,0,,,,
JNL1,03-07-2014,test,10.00,1701,0,,,,
JNL1,03-07-2014,test,-191.20,1310,0,,,,
JNL2,04-07-2014,test,49.00,8000,0,,,,
JNL2,04-07-2014,test,122.05,8001,0,,,,
JNL2,04-07-2014,test,-171.05,1310,0,,,,

After processing all lines are packed into a single journal (#822). What am I missing?

The processing output is:

Skipped header. (line 1 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 2 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Debit: 822, 03-07-2014, 8000, 0, 0, 23.3, test Date: 03-07-2014 Reference: JNL1 (line 2 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 3 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Debit: 822, 03-07-2014, 8001, 0, 0, 173.9, test Date: 03-07-2014 Reference: JNL1 (line 3 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 4 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Credit: 822, 03-07-2014, 1700, 0, 0, -16, test Date: 03-07-2014 Reference: JNL1 (line 4 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 5 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Debit: 822, 03-07-2014, 1701, 0, 0, 10, test Date: 03-07-2014 Reference: JNL1 (line 5 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 6 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Credit: 822, 03-07-2014, 1310, 0, 0, -191.2, test Date: 03-07-2014 Reference: JNL1 (line 6 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 7 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Debit: 822, 04-07-2014, 8000, 0, 0, 49, test Date: 04-07-2014 Reference: JNL2 (line 7 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 8 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Debit: 822, 04-07-2014, 8001, 0, 0, 122.05, test Date: 04-07-2014 Reference: JNL2 (line 8 in import file 'kassabonnen.csv') 
--------------------------------------------------------------------------------------------Line 9 ------------------------------------------------------------------------------------------
Added to table 'gl_trans' Credit: 822, 04-07-2014, 1310, 0, 0, -171.05, test Date: 04-07-2014 Reference: JNL2 (line 9 in import file 'kassabonnen.csv')
8 General Journals would have been successful if imported. Uncheck Trial check before importing.

Hopefully someone will be able to help me out. Thanks in advance!

Re: Import Transactions

Hi Ross,

In modules/import_transactions/includes/import_transactions.inc replace line 634:

        $sql = "SELECT id, type, trans_no, amount FROM ".TB_PREF."bank_trans ORDER BY id DESC LIMIT 1";

with

        $sql = "SELECT id, type, trans_no, amount FROM ".TB_PREF."bank_trans WHERE `type` =".$type." ORDER BY id DESC LIMIT 1";

Although ST_BANKDEPOSIT and ST_BANKPAYMENT are stored in the same table they use separate transaction numbers. I tried to import bank deposits, unfortunately the transaction id's that were assigned were already in use.

Probably line 651 must be changed accordingly to separate the sales orders and invoices, but I haven't checked that!

Re: Import Transactions

Hi Jorn,
Yes I agree. Thanks for that. The code on lines 634 and 651 need to be changed to include filtering 'type' in the SELECT statement.   

Regarding the journal entry I will do a few tests when I have the time.

Re: Import Transactions

Committed in my GitHub repo.
Thanks @jornrikkers and @raddison.

Post's attachments

diff1.png 20 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Re: Import Transactions

Hello, I'm using the Import Transactions extension in order to import our company bank statement. I've followed the Front Accounting wiki and Ross's instructions regarding setup and formatting of the bank statement CSV file but as yet have had no success in importing even a simple 2 line test file with a payment and a deposit.

Here's what a trial-run import returns via the browser interface when "Deposit Processing" was selected:

Skipped header. (line 1 in import file 'AD import Mar 15 test1.csv')
--------------------------------------------------------------------------------------------Line 2 ------------------------------------------------------------------------------------------
Ignoring payment. Use same csv under payment processing.(line 2 in import file 'AD import Mar 15 test1.csv')
--------------------------------------------------------------------------------------------Line 3 ------------------------------------------------------------------------------------------
Ignoring payment. Use same csv under payment processing.(line 3 in import file 'AD import Mar 15 test1.csv')
Error: Debits do not equal credits.

and when "Payment Processing" was selected:

Skipped header. (line 1 in import file 'AD import Mar 15 test1.csv')
--------------------------------------------------------------------------------------------Line 2 ------------------------------------------------------------------------------------------
You are here payment
Ignoring deposit. Use same csv under deposit processing. (line 2 in import file 'AD import Mar 15 test1.csv')
--------------------------------------------------------------------------------------------Line 3 ------------------------------------------------------------------------------------------
Ignoring deposit. Use same csv under deposit processing. (line 3 in import file 'AD import Mar 15 test1.csv')
Error: Debits do not equal credits.


The above is based on the importing of the following CSV file which contains one deposit and one payment. I've been careful to ensure the date format (I'm using MM/DD/YYYY) is correct, account codes are formatted as text. I left the "taxtype" column blank as we are not VAT registered and I haven't set up a 0% tax type. I also left "dim1ref", "dim2ref" and Branch_id" columns empty. It feels as if payment and receipt amounts are being overlooked at each import attempt-

reference,date,memo,payment,receipt,accountcode,taxtype,dim1ref,dim2ref,person_type_id (supplier=1 customer=2),person_id,Branch_id
Test001,03/02/2015,Boris_accom, ,75,4010,,,,2,1,
Test002,03/02/2015,Creme_de_La_pmt,50, ,5013,,,,1,1,

I've tried doing live runs with the same file but can detect no evidence of the transactions in any GL or bank account.

I'd really appreciate some help with this.

Re: Import Transactions

Have a look at the templates folder in the said module. There are 9,10,11 or 24 commas in each field in the CSV files there.

Your data set seems to be following the templates/bank_en_GB.csv file.

Which CoA are you using? The UK CoA has some errors in field order.

Check out the corrected extensions and CoAs in my unofficial GitHub Repo.

Re: Import Transactions

Hello apmuthu,

Thanks for the suggestions; they have helped me do a successful trial import of a receipt and a payment.

My data is in the templates/bank_en_GB.csv format where there are 11 commas per line. Whilst counting commas in the CSV file in my last post I noticed there was a space between the commas representing the empty payment and receipt field on the 2 data lines. When I removed the spaces as below it imported fine:

reference,date,memo,payment,receipt,accountcode,taxtype,dim1ref,dim2ref,person_type_id (supplier=1 customer=2),person_id,Branch_id
Test001,03/02/2015,Boris_accom,,75,4010,,,,2,1,
Test002,03/02/2015,Creme_de_La_pmt,50,,5013,,,,1,1,

as compared to my earlier test file with a space between the commas before the "75" in line 2 and after "50" in line 3, which failed:

reference,date,memo,payment,receipt,accountcode,taxtype,dim1ref,dim2ref,person_type_id (supplier=1 customer=2),person_id,Branch_id
Test001,03/02/2015,Boris_accom, ,75,4010,,,,2,1,
Test002,03/02/2015,Creme_de_La_pmt,50, ,5013,,,,1,1,

Here is the response for the receipt processing (payment response was also successful):

Skipped header. (line 1 in import file 'AD import Mar 15 test1.csv')
--------------------------------------------------------------------------------------------Line 2 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 12, 03/02/2015, Test001, 4010, 0, 0, 75 (line 2 in import file 'AD import Mar 15 test1.csv')
Added to table 'gl_trans Credit:': 12, 03/02/2015, 4010, 0, 0, -75, Boris_accom (line 2 in import file 'AD import Mar 15 test1.csv')
Added to table 'debtor_trans Credit: Payment from customer': 75 (line 2 in import file 'AD import Mar 15 test1.csv')
Added to table 'gl_trans' Debit bank account: 12, 03/02/2015, 1060, 0, 0, 75, Boris_accom (line 2 in import file 'AD import Mar 15 test1.csv')
--------------------------------------------------------------------------------------------Line 3 ------------------------------------------------------------------------------------------
Ignoring payment. Use same csv under payment processing.(line 3 in import file 'AD import Mar 15 test1.csv')
1 Deposits would have been successful if imported. Uncheck Trial check before importing.

The offending spaces were introduced during my formatting of the CSV file my bank lets me download. It comes in the form Date, Memo and and Amount. The Amount value is in one column with positive values for receipts and negative values for payments. I used an IF-THEN-ELSE spreadsheet function in order to create two positive valued columns from the combined +/- column. For payments: =IF(C2<0,-C2," ") took a negative payment value and made it positive or left what I thought was a blank cell but I suppose must have resulted in a space because of the space I had left between the pair of double inverted commas, " ". This went on as a space between the commas.

To answer your last question: I'm using the FA default CoA with a few custom codes that are numerically close to similar categories in the default set up. This seems to suit my purposes fine.

47 (edited by apmuthu 08/27/2015 05:06:49 pm)

Re: Import Transactions

@gbw: Thankyou for your valuable feedback regarding spaces.

Since the strings in the CSV are not quoted or double quoted, spaces should be taken as a zero for numerical fields and trimmed for others in the extension.

Wiki-ed it.

Re: Import Transactions

Hey I am trying to use this module and having an issue with importing transactions.  The issue I am having has to do with my COA numbers I have created.  my account numbers are formatted like this: 5-00-000-01-01.  The import module won't recognize that kind of COA number, but if I put in a 4 digit COA number everything imports properly.

Can someone tell me how to make it work so my COA numbers will work properly?  Thanks.

Re: Import Transactions

myhandgunpursedotcom wrote:

Hey I am trying to use this module and having an issue with importing transactions.  The issue I am having has to do with my COA numbers I have created.  my account numbers are formatted like this: 5-00-000-01-01.  The import module won't recognize that kind of COA number, but if I put in a 4 digit COA number everything imports properly.

Can someone tell me how to make it work so my COA numbers will work properly?  Thanks.

Also, here is what I get when I run the trail run on my cvs file:
Skipped header. (line 1 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 2 ------------------------------------------------------------------------------------------
Error: Account code 5-00-000-01-01 does not exist
--------------------------------------------------------------------------------------------Line 3 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 15, 11/11/2016, JL25, 5700, 0, 0, -411.88 (line 3 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 15, 11/11/2016, 5700, 0, 0, 411.88, General Retail (line 3 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -411.88 (line 3 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 15, 11/11/2016, 1060, 0, 0, -411.88, General Retail (line 3 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 4 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 16, 11/12/2016, JL26, 5100, 0, 0, -272.45 (line 4 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 16, 11/12/2016, 5100, 0, 0, 272.45, Fuel/Auto (line 4 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -272.45 (line 4 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 16, 11/12/2016, 1060, 0, 0, -272.45, Fuel/Auto (line 4 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 5 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 17, 11/13/2016, JL27, 5100, 0, 0, -12.26 (line 5 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 17, 11/13/2016, 5100, 0, 0, 12.26, Fuel/Auto (line 5 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -12.26 (line 5 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 17, 11/13/2016, 1060, 0, 0, -12.26, Fuel/Auto (line 5 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 6 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 18, 11/14/2016, JL28, 5100, 0, 0, -357.98 (line 6 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 18, 11/14/2016, 5100, 0, 0, 357.98, Fuel/Auto (line 6 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -357.98 (line 6 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 18, 11/14/2016, 1060, 0, 0, -357.98, Fuel/Auto (line 6 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 7 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 19, 11/15/2016, JL29, 5700, 0, 0, -1000 (line 7 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 19, 11/15/2016, 5700, 0, 0, 1000, Transportation (line 7 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -1000 (line 7 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 19, 11/15/2016, 1060, 0, 0, -1000, Transportation (line 7 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 8 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 20, 11/16/2016, JL30, 5100, 0, 0, -14 (line 8 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 20, 11/16/2016, 5100, 0, 0, 14, Loading/Unloading (line 8 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -14 (line 8 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 20, 11/16/2016, 1060, 0, 0, -14, Loading/Unloading (line 8 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 9 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 21, 11/17/2016, JL31, 5700, 0, 0, -50 (line 9 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 21, 11/17/2016, 5700, 0, 0, 50, General Retail (line 9 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -50 (line 9 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 21, 11/17/2016, 1060, 0, 0, -50, General Retail (line 9 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 10 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 22, 11/18/2016, JL32, 5020, 0, 0, -89.06 (line 10 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 22, 11/18/2016, 5020, 0, 0, 89.06, Reefer Fuel (line 10 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -89.06 (line 10 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 22, 11/18/2016, 1060, 0, 0, -89.06, Reefer Fuel (line 10 in import file 'CSV_484074944.csv')
--------------------------------------------------------------------------------------------Line 11 ------------------------------------------------------------------------------------------
Added to table 'bank_trans': 23, 11/19/2016, JL33, 5100, 0, 0, -491.68 (line 11 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Debit: 23, 11/19/2016, 5100, 0, 0, 491.68, Fuel/Auto (line 11 in import file 'CSV_484074944.csv')
Added to table 'supp_trans' Debit: Supplier paid -491.68 (line 11 in import file 'CSV_484074944.csv')
Added to table 'gl_trans' Credit bank account: 23, 11/19/2016, 1060, 0, 0, -491.68, Fuel/Auto (line 11 in import file 'CSV_484074944.csv')
1 error(s) detected. Correct before importing.

When you Look at what it pulls up below all that, it shows the correct account number and account name.  I know FA doesn't like account numbers bigger than like 8 digits and I had to already make other modifications to the core to make my longer account numbers work in the program.

Re: Import Transactions

It is possible that the module's files are accepting only integers as account numbers.