Bash trim 或 sed 换行符

Question

日志文件中有一些新行要删除（不是全部）；已经尝试了很多 trim 的方法，但没有成功（通常以一大行结尾）。

我想我需要进行特定的搜索，当 trimming 或 sedding 时，但不确定。

示例文件：

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams)
   at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams)
   at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

需要：

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

非常感谢任何帮助！

Answer 1

你可以用 awk 做到这一点：

awk 'BEGIN {ORS=""} /^ /{ print [=10=]; next;} {print "\n"[=10=];}' input_file | sed 's/\r//g'

这给出了

JOBV2|MISC: PURCHASE_DENIED|Puchasemode = sync ; iEoCode = 992 ; teminalId = XXXXXXXXXXXX; offeingId = XXXXXX; entalType = ps ; pice = 1649 ; entalWindowHs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TansactionID = IsUG=N;TId=X;VID=XXXX;Pice=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTansactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPice=16.49;DisPice=0.00;NetPice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PuchasePaams objPuchasePaams)   at STRING.JOBV3(Sting stInputPaams)|stInputPaams = stUnitAddess=XXXXXXXXXXXX,teminalid=XXXXXXXXXXXX,offeingid=XXXXXX,entalType=PS,pice=1649,entalWindowHs=0,IsAdult=false,IsRedeemTitle=false,pomotionType=,isUpgade=N,catalogpice=0,stpomotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PuchasePaams objPuchasePaams)   at STRING.JOBV3(Sting stInputPaams)|stInputPaams = stUnitAddess=XXXXXXXXXXXX,teminalid=XXXXXXXXXXXX,offeingid=XXXXXX,entalType=PS,pice=1649,entalWindowHs=0,IsAdult=false,IsRedeemTitle=false,pomotionType=,isUpgade=N,catalogpice=0,stpomotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Puchasemode = sync ; iEoCode = 992 ; teminalId = XXXXXXXXXXXX ; offeingId = XXXXXX; entalType = ps ; pice = 1649 ; entalWindowHs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TansactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pice=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTansactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPice=16.49;DisPice=0.00;NetPice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Puchase_Denied Eo sting: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

替代方法：

awk 'BEGIN {ORS=""} /\)$/{print [=12=]; next;} {print [=12=]"\n";}' input_file | sed 's/\r//g'

Answer 2

另一种解决方法，可以使用sed

sed -r ':a;N;$!ba;s/\n\s+/ /g' file

或者，不使用扩展正则表达式选项 -r

sed ':a;N;$!ba;s/\n[ \t][ \t]*/ /g' file

你得到

JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=X;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457
JOBV3|MISC: PURCHASE FAILED: PURCHASE_DENIED   at STRING.JOBV2(PurchaseParams objPurchaseParams) at STRING.JOBV3(String strInputParams)|strInputParams = strUnitAddress=XXXXXXXXXXXX,terminalid=XXXXXXXXXXXX,offeringid=XXXXXX,rentalType=PS,price=1649,rentalWindowHrs=0,IsAdult=false,IsRedeemTitle=false,promotionType=,isUpgrade=N,catalogprice=0,strpromotionid=0; ; |2015-10-07 00:00:14.320
JOBV2|MISC: PURCHASE_DENIED|Purchasemode = sync ; iErrorCode = 992 ; terminalId = XXXXXXXXXXXX ; offeringId = XXXXXX; rentalType = ps ; price = 1649 ; rentalWindowHrs = 0 ; IsAdult = False ; IsRedeemTitle = False ; TransactionID = IsUG=N;TId=XXXXXXXXXXXX;VID=XXXX;Price=16.49;Pid=0;;CP=0 ; RedeemPoints =  ; PointsRatio =  ; TotalMRPoints =  ; NewTransactionID = IsUG=N;TId=XXXXXXXXXXXX:VID=XXXX;Pid=0;RetPrice=16.49;DisPrice=0.00;NetPrice=16.49;MRPoints=0;P2DVal=0;TotalPoints=0; ; |2015-10-07 00:00:09.313
JOBV2|MISC: Non-Purchase_Denied Error string: BILLING SYSTEM IS DOWN|^@|2015-10-07 00:02:13.457

Answer 3

如果您正在做的是将以空格开头的行连接到上一行，您可以在 bash.

中执行此操作

last=""
while IFS="" read line; do
  if [[ "$line" =~ ^[[:space:]] ]]; then
    echo -n "$last "
  else
    echo "$last"
  fi
  last="$line"
done < inp
echo "$line"

甚至在POSIX（非bash）shell:

last=""
while IFS="" read line; do
  if expr "$line" : '[[:space:]]' >/dev/null; then
    echo -n "$last "
  else
    echo "$last"
  fi
  last="$line"
done < inp
echo "$line"

Bash trim 或 sed 换行符

Bash trim or sed newline

linux

awk

newline

sed

trim