Home » SQL & PL/SQL » SQL & PL/SQL » External table not fully loaded (12c,windows NT)
External table not fully loaded [message #651040] Tue, 10 May 2016 07:46 Go to next message
pstanand
Messages: 133
Registered: February 2005
Location: Chennai,India
Senior Member
Hi,
I have a txt file delimited with pipe and having 432,333 rows of records.
I use the following external table script to read the txt file.
External table is created successfully with records.
But the count is only 413,098 instead 432,333.
There is not bad records created or discarded file created.

could you please let me know what could be the reason and how can this be resolved?

thanks.
create table sap_prdt_tst
    (
MATERIAL_NUMBER varchar2(20),
WEIGHT_UNIT varchar2(5),
WARRANTY_CODE varchar2(5),
NET_WEIGHT number(20,3),
PRODUCT_LINE varchar2(5),
EXPORT_CON_CLASS varchar2(17),
OBS_COMMENT varchar2(255),
PRODUCT_HIERACHY varchar2(10),
MATERIAL_GROUP varchar2(10),
LASTUPDATE varchar2(29)
)
   organization external
   (type oracle_loader
    default directory MIGRATION_DIR
    access parameters
    (
     records delimited BY '\n'
     BADFILE MIGRATION_DIR:'prdts_xt.bad'
     LOGFILE MIGRATION_DIR:'prdts_xt.log'
     NODISCARDFILE
       CHARACTERSET UTF8
       STRING SIZES ARE IN CHARACTERS 
           READSIZE 134217728
           NOBADFILE
           NODISCARDFILE
           NOLOGFILE
           SKIP 1
     fields terminated by '|'
     missing field values are null
     (
     MATERIAL_NUMBER char(20),
WEIGHT_UNIT char(5),
WARRANTY_CODE char(5),
NET_WEIGHT char(20),
PRODUCT_LINE char(5),
EXPORT_CON_CLASS char(17) ,
OBS_COMMENT char(300),
PRODUCT_HIERACHY char(10),
MATERIAL_GROUP char(10),
LASTUPDATE char(29)date_format date mask "yyyy-mm-dd"
)
   )
    location ('sap_products.txt')
   )
   parallel 4
   reject limit unlimited
   nomonitoring;
Re: External table not fully loaded [message #651041 is a reply to message #651040] Tue, 10 May 2016 08:07 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
you post this situation exists on Windows system where text line terminator is "CR/LF", but you code Oracle EXTERNAL TABLE to only use "records delimited BY '\n'"
What OS originally produce this text file?
Re: External table not fully loaded [message #651046 is a reply to message #651041] Tue, 10 May 2016 08:17 Go to previous messageGo to next message
pstanand
Messages: 133
Registered: February 2005
Location: Chennai,India
Senior Member
Really sorry the oracle server is resides on Unix OS.
Re: External table not fully loaded [message #651048 is a reply to message #651040] Tue, 10 May 2016 08:23 Go to previous messageGo to next message
John Watson
Messages: 8938
Registered: January 2010
Location: Global Village
Senior Member
You have these lines

BADFILE MIGRATION_DIR:'prdts_xt.bad'
LOGFILE MIGRATION_DIR:'prdts_xt.log'
NODISCARDFILE

then these lines

NOBADFILE
NODISCARDFILE
NOLOGFILE

so you don't actually know what is going on. You need to enable all three files.
Re: External table not fully loaded [message #651051 is a reply to message #651048] Tue, 10 May 2016 08:55 Go to previous messageGo to next message
pstanand
Messages: 133
Registered: February 2005
Location: Chennai,India
Senior Member
Hi,
I have enabled the log and bad files in my code. And have attached my log file.
The lastupdate column in my table is null for the records which is missing in the external table.
The lastupdate column since it is NULL comes with 0 value and so it is not gone into the external table.
Any suggestion on how to handle this please?

create table sap_prdt_tst
    (
      MATERIAL_NUMBER varchar2(70),
      WEIGHT_UNIT varchar2(5),
      WARRANTY_CODE varchar2(5),
      NET_WEIGHT number(20,3),
      PRODUCT_LINE varchar2(5),
      EXPORT_CON_CLASS varchar2(20),
      OBS_COMMENT varchar2(355),
      PRODUCT_HIERACHY varchar2(10),
      MATERIAL_GROUP varchar2(10),
      LASTUPDATE varchar2(29)
)
   organization external
   (type oracle_loader
    default directory MIGRATION_DIR
    access parameters
    (
     records delimited BY '\n'
     BADFILE MIGRATION_DIR:'prdts_xt.bad'
     LOGFILE MIGRATION_DIR:'prdts_xt.log'
     NODISCARDFILE
     CHARACTERSET UTF8
     STRING SIZES ARE IN CHARACTERS 
     READSIZE 134217728
     SKIP 1
     fields terminated by '|'
     missing field values are null
     (
     MATERIAL_NUMBER char(70),
     WEIGHT_UNIT char(5),
     WARRANTY_CODE char(5),
     NET_WEIGHT char(20),
     PRODUCT_LINE char(5),
     EXPORT_CON_CLASS char(20) ,
     OBS_COMMENT char(355),
     PRODUCT_HIERACHY char(10),
     MATERIAL_GROUP char(10),
     LASTUPDATE char(29)date_format date mask "yyyy-mm-dd"
)
   )
    location ('sap_products.txt')
   )
   parallel 4
   reject limit unlimited
   nomonitoring;
  • Attachment: prdts_xt.log
    (Size: 853.61KB, Downloaded 1187 times)
Re: External table not fully loaded [message #651054 is a reply to message #651051] Tue, 10 May 2016 10:20 Go to previous messageGo to next message
John Watson
Messages: 8938
Registered: January 2010
Location: Global Village
Senior Member
Quote:
Any suggestion on how to handle this please?

Yes, instruct whomever is generating file to produce valid data.

By the way, you might want to say "thank you for your help".
Re: External table not fully loaded [message #651089 is a reply to message #651054] Wed, 11 May 2016 01:43 Go to previous messageGo to next message
pstanand
Messages: 133
Registered: February 2005
Location: Chennai,India
Senior Member
Hi
could you please help me here to handle this issue?

thanks.

Regards,
pstanand.
Re: External table not fully loaded [message #651090 is a reply to message #651089] Wed, 11 May 2016 01:46 Go to previous messageGo to next message
John Watson
Messages: 8938
Registered: January 2010
Location: Global Village
Senior Member
I've already told you what I would do: instruct the person giving you this file to load it correctly. If I understand your post correctly, there are rows where the last column has a zero. Zero is not a valid date.

Re: External table not fully loaded [message #651108 is a reply to message #651040] Wed, 11 May 2016 07:54 Go to previous message
pstanand
Messages: 133
Registered: February 2005
Location: Chennai,India
Senior Member
Hi,
thanks for your suggestion. I have received the file in correct format and my external table script works fine now.

Appreciate your help.

Regards,
pstanand
Previous Topic: Query Analytics
Next Topic: Partition taking more space
Goto Forum:
  


Current Time: Sun Jun 30 17:05:12 CDT 2024