Logstash 模式匹配超过它应该的

Question

我有一个日志行（多行匹配），如下所示：

3574874    14/Jul/2016 20:42:37 +0000  ERROR [http-bio-0.0.0.0-8443-exec-128] error_jsp            _jspService    > could not lock: [com.myCompany.myProject.bean.scheduling.Area#6306]; SQL [/* UPGRADE lock com.myCompany.myProject.bean.scheduling.Area */ select Area_id from Area_ID where Area_id =? and Area_Version =? for update]; nested exception is org.hibernate.exception.LockAcquisitionException: could not lock: [com.myCompany.myProject.bean.scheduling.Area#6306]
org.springframework.dao.CannotAcquireLockException: could not lock: [com.myCompany.myProject.bean.scheduling.MyClass#6306]; SQL [/* UPGRADE lock com.myCompany.myProject.bean.scheduling.MyClass */ select Area_ID from Area where Area =? and Area =? for update]; nested exception is org.hibernate.exception.LockAcquisitionException: could not lock: [com.myCompany.myProject.bean.scheduling.Area#6306]
    at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:639)
    at org.springframework.orm.hibernate3.HibernateExceptionTranslator.convertHibernateAccessException(HibernateExceptionTranslator.java:89)
    at org.springframework.orm.hibernate3.HibernateExceptionTranslator.translateExceptionIfPossible(HibernateExceptionTranslator.java:68)
    at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:58)
    at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:163)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:633)
    at com.myCompany.myProject.dal.hibernate.Impl$$EnhancerBySpringCGLIB$5be625.lock(<generated>)
    at com.myCompany.myProject.scheduling.Area.AreaClose(MyClass.java:1265)
    at com.myCompany.myProject.scheduling.Area.handleProviderDisconnect(MyClass.java:1190)
    at com.myCompany.myProject.scheduling.Area$$FastClassBySpringCGLIB$0c3d67.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)

我的模式文件是这样安排的：

    # AW Tomcat formatting
    TOMCAT_DATE %{MONTHDAY}[./-]%{MONTH}[./-]%{YEAR}
    TOMCAT_TS %{BASE10NUM}
    #%{BASE10NUM}
    TOMCAT_TIME %{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])
    TOMCAT_THREAD \[(.+?)\]
    TOMCAT_CLASS [A-Za-z0-9]+
    TOMCAT_CLASS_METHOD %{NOTSPACE}
    TOMCAT_TIMESTAMP %{TOMCAT_DATE}[\s]+%{TOMCAT_TIME}
    TOMCAT_MESSAGE .+
    TOMCAT_IP %{IP}
    TOMCAT_DATA %{NOTSPACE}

    TOMCAT_LOG (?:%{TOMCAT_TS:log_ts})[\s]+(?:%{TOMCAT_TIMESTAMP:log_timestamp})[\s]+%{INT}[\s]+(?:%{LOGLEVEL:log_level})[\s]+(?:%{TOMCAT_THREAD:thread})[\s]+(?:%{TOMCAT_CLASS:class_name})[\s]$

当我在 logstash 中查看它时，似乎线程的代码匹配的不仅仅是线程......它一直匹配到 [com.myCompany.myProject.bean.scheduling.MyClass#6306]；

当我只获取 space 的线程和正则表达式并将其扔到 regexr 或 regex101 上时，我无法使用以下方法重新创建它：

\[(.+?)\][\s]+

有谁知道为什么正则表达式在除 logstash 之外的所有地方都有效？此外，这也适用于大约 99.5% 的传入 tomcat 日志....

Answer 1

您的日志中有 +。您需要将其与 \+ 匹配，或将其设为可选 \+?:

TOMCAT_LOG (?:%{TOMCAT_TS:log_ts})\s+(?:%{TOMCAT_TIMESTAMP:log_timestamp})\s+\+?%{INT}\s+(?:%{LOGLEVEL:log_level})\s+(?:%{TOMCAT_THREAD:thread})\s+(?:%{TOMCAT_CLASS:class_name})\s*$
                                                                             ^^^

Logstash 模式匹配超过它应该的

Logstash pattern matching more than it should

regex

pattern-matching

grok